FAQ

Here are some of the questions about PureConfig frequently asked on issues and other channels:

How can I use PureConfig with Spark 2.1.0 (problematic shapeless dependency)?

Apache Spark (specifically version 2.1.0) has a transitive dependency on shapeless 2.0.0. This version is too old to be used by PureConfig, making your Spark project fail when using spark-submit. The solution is to shade, i.e. rename, the version of shapeless used by PureConfig.

SBT

If you are using the sbt-assembly plugin to create your JARs you can shade shapeless by adding to your assembly.sbt file the following setting:

assembly / assemblyShadeRules := Seq(ShadeRule.rename("shapeless.**" -> "new_shapeless.@1").inAll)

Maven

The maven-shade-plugin can shade shapeless by adding to your pom.xml file the following block:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>3.0.0</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <createDependencyReducedPom>false</createDependencyReducedPom>
        <relocations>
            <relocation>
                <pattern>shapeless</pattern>
                <shadedPattern>shapelesspureconfig</shadedPattern>
            </relocation>
        </relocations>
    </configuration>
</plugin>

How can I avoid displaying sensitive parts of my configuration, such as passwords, when I print the result of PureConfig?

Configuration commonly includes sensitive information, such as passwords or keys. Generally these values should not appear in logging.

Unfortunately this means that Scala’s typically helpful behavior of generating transparent toString methods gets in the way. When a configuration is loaded into a case class, then the default toString generated by Scala will display sensitive information. This behavior is standard Scala and has nothing to do with PureConfig per se.

A solution we’ve seen address this concern is to wrap the sensitive field in an AnyVal new type which simply redefines toString. Please see the example below:

import pureconfig._
import pureconfig.generic.auto._

case class Sensitive(value: String) extends AnyVal {
  override def toString: String = "MASKED"
}

case class SuperSecretConfig(
  username: String,
  password: Sensitive,
  apiKey: Sensitive)

Then when we print out the SuperSecretConfig after loading it via PureConfig, the sensitive values are masked:

val secret = ConfigSource.string("""{
  username: john.smith
  password: password123
  api-key: 8ef72f48-2143-48af-9573-3b519bbcb777
}""").loadOrThrow[SuperSecretConfig]
// secret: SuperSecretConfig = SuperSecretConfig(
//   "john.smith",
//   Sensitive("password123"),
//   Sensitive("8ef72f48-2143-48af-9573-3b519bbcb777")
// )

println(secret)
// SuperSecretConfig(john.smith,MASKED,MASKED)

But, of course, the values we need are still there:

secret.password.value 
// res2: String = "password123"
secret.apiKey.value
// res3: String = "8ef72f48-2143-48af-9573-3b519bbcb777"

How do I debug “implicit not found” errors?

When we want to load a config with a large structure and Scala refuses to compile a reader for it, it can be difficult to understand the reason of the failure. Consider the following example:

import pureconfig._
import pureconfig.generic.auto._

// not a case class
class Custom(x: Int, s: String) {
  def asString = s"x = $x, s = $s"
}
// not a case class
class Custom2(x: Int, s: String) {
  def asString = s"x: $x, s: $s"
}

sealed trait Conf
case class ConfA(a: Boolean, b: Option[Boolean]) extends Conf
sealed trait ConfB extends Conf
case class ConfB1(a: Int) extends ConfB
case class ConfB2(a: String) extends ConfB
case class ConfC(a: Option[Custom], b: Custom2) extends Conf

When we try to load a Conf from a config, we’ll simply get this error message:

ConfigSource.default.load[Conf]
// error: could not find implicit value for parameter reader: pureconfig.ConfigReader[MdocApp1.this.Conf]

In PureConfig, the derivation of config readers and writers is done by chaining implicits - the converters of larger structures (like Conf) depend on the implicit converters of smaller ones (like Boolean or Custom). However, the Scala compiler for Scala 2.x is not helpful by default in case one of those upstream dependencies is missing, limiting itself to showing the message above.

To more efficiently debug these “implicit not found” errors, we recommend using splain. splain was integrated into the compiler in Scala 2.13.6 and can be enabled with the -Vimplicits scalac flag. For Scala 2.13 versions prior to 2.13.6 or Scala 2.12 versions, please refer to the project’s documentation for instructions on how to include the compiler plugin.

When code using PureConfig derived converters is compiled using the compiler plugin recommended above, you will get a more thorough error message. In this particular case, the message will be quite large because there are some alternative implicit paths to try and the path until we arrive at an implicit not found outside PureConfig’s derivation code is big:

ConfigSource.default.load[Conf]
// [error] !I reader: ConfigReader[Conf]
// [error] shapeless.lazily.apply invalid because
// [error] !I lv: DerivedConfigReader[Conf]
// [error] ――DerivedConfigReader.productReader invalid because
// [error]   !I gen: LabelledGeneric.Aux[Conf, Repr]
// [error] ――――LabelledGeneric.materializeProduct invalid because
// [error]     !I gen: Generic.Aux[Conf, V]
// ...
// [error]   ConfigSource.default.load[Conf]
// [error]                            ^

The error message above shows that the compiler tried various alternatives to create a ConfigReader for Conf but all of them failed due to some missing implicits deeper down in the path. A way to find which implicits we failed to provide as downstream users is to look for missing ConfigReader instances deeper in the chain. One of them is shown when attempting to use ConfigReader.optionReader:

// [error] !I reader: ConfigReader[Conf]
// [error] shapeless.lazily.apply invalid because
// [error] !I lv: DerivedConfigReader[Conf]
// [error] ――DerivedConfigReader.productReader invalid because
// [error]   !I gen: LabelledGeneric.Aux[Conf, Repr]
// [error] ――――LabelledGeneric.materializeProduct invalid because
// [error]     !I gen: Generic.Aux[Conf, V]
// [error]
// ...
// [error] ――――――――――――――――――――――――――DerivedConfigReader.productReader invalid because
// [error]                           !I cc: MapShapedReader[ConfC, Repr, DefaultRepr]
// [error] ――――――――――――――――――――――――――――MapShapedReader.labelledHConsReader invalid because
// ...
// [error] ――――――――――――――――――――――――――――――――――ConfigReader.optionReader invalid because
// [error]                                   !I conv: ConfigReader[Custom]
// ...
// [error]   ConfigSource.default.load[Conf]
// [error]                            ^

This tells us that there’s a missing ConfigReader in scope for type Custom. Since Custom isn’t a case class nor a sealed trait, we’re unable to derive a ConfigReader for it using PureConfig’s generic derivation. Making Custom a case class or explicitly providing a ConfigReader instance for it helps us get rid of this error.

Once that’s fixed, we’re still left with another error. Similarly, we can look for missing ConfigReader instances deeper in the chain. We’re able to find one missing for Custom2:

// [error] !I reader: ConfigReader[Conf]
// [error] shapeless.lazily.apply invalid because
// [error] !I lv: DerivedConfigReader[Conf]
// [error] ――DerivedConfigReader.productReader invalid because
// [error]   !I gen: LabelledGeneric.Aux[Conf, Repr]
// [error] ――――LabelledGeneric.materializeProduct invalid because
// [error]     !I gen: Generic.Aux[Conf, V]
// ...
// [error] ――――――――――――――――――――――――――DerivedConfigReader.productReader invalid because
// [error]                           !I cc: MapShapedReader[ConfC, Repr, DefaultRepr]
// [error] ――――――――――――――――――――――――――――MapShapedReader.labelledHConsReader invalid because
// [error]                             !I tConfigReader: MapShapedReader[ConfC, ('b ->> Custom2) :: HNil, Option[Custom2] :: HNil]
// [error] ――――――――――――――――――――――――――――――MapShapedReader.labelledHConsReader invalid because
// [error]                               !I hConfigReader: ConfigReader[Custom2]
// ...
// [error]   ConfigSource.default.load[Conf]
// [error]                            ^

Custom2 isn’t also a case class nor a sealed trait, so either making it a case class or explicitly providing a ConfigReader instance would fix our error.