Introducing JUnit 5 FormattedSource
Quite some time ago, JUnit’s @CsvAnnotation caught my attention. It turned out so interesting, that I dedicated it a separate blog post and a significant part of a conference talk. It also inspired me to create a new way of writing parameterized tests with JUnit 5.
The JUnit 5 FormattedSource library allows defining test case arguments in a human-readable way, following a user-defined format. As a result, it can be used to improve tests readability. Let’s have a look at an example:
class CalculatorTest {
private final Calculator calculator = new Calculator();
@FormattedSourceTest(format = "{0} + {1} = {2}", lines = {
"1 + 2 = 3",
"3 + 4 = 7"
})
void calculatesSum(int a, int b, int expectedSum) {
Assertions.assertEquals(expectedSum, calculator.sum(a, b));
}
}
The test execution output shows another benefit of using FormattedSource - the library customizes displayed test case names for us with zero additional effort:
calculatesSum(int, int, int) ✔
├─ 1 + 2 = 3 ✔
└─ 3 + 4 = 7 ✔
Of course, it can do a lot more than this 😉
This post won’t exhaustively explain what’s possible with FormattedSource. That’s what its User Guide was created for. Instead, I’d like to show what problems the library can solve and how certain things work under the hood. At the same time, it will be a celebration of the recent 1.0.0 release.
Why?
Probably the most surprising usage of the @CsvSource I came up with was using a multi-character string as a field delimiter. Although it felt questionable from the CSV format perspective, it worked just as expected:
@ParameterizedTest
@CsvSource(delimiterString = "maps to", textBlock = """
'foo' maps to 'bar'
'junit' maps to 'jupiter'
""")
void shouldMapPair(String input, String expected) {
String actual = pairMapper.map(input);
Assertions.assertEquals(expected, actual);
}
I found this way of defining test cases really powerful. If only it supported multiple delimiterString at a time, we could use it with more complex formats like:
<input> maps to <output> using rule <ruleId>
Yet, it does not.
Using @CsvSource for defining tests like that felt a bit hacky from the beginning. Instead of further abusing it, I decided to create an alternative. That’s how the FormattedSource was born.
The FormattedSource library allows defining test cases in a simple and flexible way. By specifying the format string, we’re defining how all our test case definitions will look like. Together with a rather relaxed approach to handling additional whitespaces, this allows us to focus on the readability. What was impossible to achieve with @CsvSource has become a standard solution:
@FormattedSourceTest(format = "{0} maps to {1} using rule {2}", textBlock = """
'foo' maps to 'bar' using rule 486
'junit' maps to 'jupiter' using rule 44
""")
void mapsOneValueToAnother(String input, String mapped, int ruleId) {
// ...
}
Apart from test case argument placeholders, the format string has no special meaning. In other words, you can define it in the most readable way each time. Instead of referring to the arguments by their indexes (like for the built-in @DisplayName annotation), you can also define your own argumentPlaceholder:
@FormattedSourceTest(
format = "replaces ? with ?", argumentPlaceholder = "?",
textBlock = """
replaces foo with bar
""")
void replacesOneStringWithAnother(String toBeReplaced, String replacement) {
// toBeReplaced == "foo", replacement == "bar"
}
Two annotations
Using a single @FormattedSourceTest annotation may look somewhat weird to those who worked with JUnit 5 parameterized tests before. Where is the @ParameterizedTest? How does it manipulate the displayed test case names?
There’s no magic here. Our first example could be also implemented like this:
class CalculatorTest {
private final Calculator calculator = new Calculator();
@ParameterizedTest(name = "{0} + {1} = {2}")
@FormattedSource(format = "{0} + {1} = {2}", lines = {
"1 + 2 = 3",
"3 + 4 = 7"
})
void calculatesSum(int a, int b, int expectedSum) {
Assertions.assertEquals(expectedSum, calculator.sum(a, b));
}
}
@FormattedSource it’s just a custom parameterized test argument source, and you can always use it like that. In fact, both @FormattedSourceTest and @FormattedSource annotations share the identical set of arguments, so they could be used interchangeably.
The benefit of combining two rather standard annotations into one is its expressiveness. Because the format string would always represent a human-readable string, it feels like the right candidate for the displayed test case name. Using @FormattedSourceTest allows you to avoid repeating the format string in the @ParameterizedTest#name.
Both annotatations support many features of the built-in @CsvSource, like declaring test cases using Java 15+ Text Blocks (instead of arrays of lines) and defining your own null values. This reduces the onboarding time and makes the API a bit more familiar.
When?
Using FormattedSource could be a good idea when expressing test cases in a human-readable way makes sense. You can think of it as a competitor to the @CsvSource when the number of arguments is reasonable and they can be easily represented as relatively short strings.
FormattedSource works well for testing the mapping code like mappers and encoders. We could then use a simple arrow in the format string:
@FormattedSourceTest(
format = "{0} -> {1}",
textBlock = """
'foo' -> 'bar'
'junit' -> 'jupiter'
"""
)
void mapsOneValueToAnother(String input, String expectedValue) { ... }
or replace it with a more descriptive string like {0} maps to {1}. Alternatively, we can make the whole test case definition sound like a real sentence:
@FormattedSourceTest(
format = "encodes {0} seconds as {1}",
lines = {
"encodes 15 seconds as 'PT15S'",
"encodes 180 seconds as 'PT3M'",
"encodes 172800 seconds as 'PT48H'"
}
)
void encodesDurationAsIso8601(long seconds, String expected) { ... }
Of course, we’re not limited only to the mapping code and passing strings as the arguments. The FormattedSource is just a JUnit 5 argument source, so it can benefit from the built-in argument conversions. This allows applying a far more creative approach - for example, to test our HTTP API:
enum UserType { CUSTOMER, ADMIN }
@FormattedSourceTest(
format = "{0} returns {1} for {2}",
textBlock = """
/api/items returns 200 for CUSTOMER
/api/items returns 404 for ADMIN
/api/admin/inventory returns 401 for CUSTOMER
/api/admin/inventory returns 200 for ADMIN
"""
)
void protectsEndpointsBasedOnUserType(URI uri, int httpCode, UserType type) {
User user = TestUserFactory.authenticated(type);
testHttpClient.get(uri).returnsResponseCode(httpCode);
}
However, this does not mean that you should forget about other argument sources! When the number of arguments or test cases is high, introducing a human-readable format may be impractical. In these cases, @CsvSource and the lesser known @CsvFileSource annotation could be a better choice. Additionally, the FormattedSource won’t help when the argument can’t be easily represeted as string.
Some nerdy details
Creating FormattedSource library was a fascinating journey. Apart from its functionality, there were many things around it that required my attention. Luckily, I didn’t have to come up with everything on my own. The amazing “Maintaining a medium-sized Java library in 2022 and beyond” guide by Michael Simons was a priceless source of inspiration.
FormattedSource uses JReleaser to automate some of the release-related activities, including:
- uploding the generated artifact to Maven Central (which is still a bit cumbersome the first time),
- generating changelog from the commit history (using Conventional Commits specification),
- creating GitHub release.
The rest is handled by a single, good-old Bash script.
The code has been organized into a multi-module Maven project. This wouldn’t deserve a mention here without the support of the Java Platform Module System (JPMS). At the time of writing, the library has two submodules:
junit5-formatted-source-parent
+- junit5-formatted-source
| +- src (library code)
| \- test (unit, classpath tests)
\- junit5-formatted-source-tests
\- test ("integration", module path tests)
junit5-formatted-source-tests includes all the “integration” tests that verify annotation behavior. This Maven submodule has its own module-info.java file explicitly importing com.mikemybytes.junit5.formatted as the dependency. As a result, it allows me to tests the JPMS support along with the functionality. It also serves as a living example of how to use the library when running on module path.
Version 1.0.0 still requires at least Java 11 to run. As much as I advertise running on at least the latest LTS version available, I also want to avoid closing the doors for the late adopters. Also, sticking to the same baseline version as Spring Boot 3.0 and Quarkus 3.0 felt like a reasonable idea. At the same time, Java 17+ is required in the junit5-formatted-source-tests module, so it’s possible to test newer language features like Text Blocks there.
Summary
The FormattedSource library brings another flavor to writing parameterized tests with JUnit 5. It’s a simple but also quite powerful way of defining test cases in a human-readable way.
The recent 1.0.0 release aims to emphasise the API stability and overall project maturity. Yet, providing all the initially envisioned features doesn’t prevent from further enhancements. The library remains open for contributions and feedback.
P.S. Feel free to leave a star on the project’s repo in case you liked it! ⭐