Create an isolation testing environment

Writing test code when creating software is a great way to increase the reliability of your service and reduce side effects.

Creating an “isolated test environment” is a very important concept in testing, and here’s how you can build one.

All example code is on Github repo.

What is Isolation testing?

Isolation testing is the process of decomposing a system into multiple modules so that they can be easily tested or evaluated.

By decomposing a system into modules, we can focus on testing individual features or methods “unaffected by complexity and dependencies”.

It’s a bit abstract to explain, so let’s imagine a situation.

What happens when you’re not isolated

First, when you save a Product in the first test(testSaveProduct), the test succeeds without throwing an exception.

Then, when the second test(testFindProductThrowsException) is run, test fails because the ‘NoExistProductException’ exception is not thrown because the information for the product registered in the first test already exists in the DataBase.

Because each test was not isolated, the result of one test affected the result of another test.

On the other hand, run the tests one by one the tests will succeed because the information registered in the DataBase is not duplicated.

In this way, tests that do not always guarantee the same result due to reasons such as the order of execution in tests that use a shared resource such as the DataBase are called `Non-Determinisitic Tests’.

Non-deterministic tests are contagious, so if you have a collection of 100 tests with 10 non-deterministic tests, they will occasionally fail, which will ruin the whole test suite.

How isolate test?

The key to test isolation is that they must be executed independently and deterministically, regardless of order.

Here’s how to create an isolated test environment, according to [Martin Fowler’s blog]((https://martinfowler.com/articles/nonDeterminism.html).

rebuild starting state from scratch, or ensure that each test cleans up properly after itself.
when you’re using databases, is to conduct your tests inside a transaction, and then to rollback the transaction at the end of the test.

Several frameworks provide functionality to do this, let’s take a look at them one by one.

@BeforeEach, @AfterEach

@BeforeEach and @AfterEach annotations are features supported by the Junit framework that, when specified in a method, indicate that it should be executed before or after each method in the current test class, respectively.

BeforeEach(setUp) can specify what a method inside that class always does before it starts, in this case, we’ve defined the fixtures we need for the test so that it starts by creating two Products and testing them before running the test.

**AfterEach(tearDown) is responsible for destroying the data after the test is finished so that other tests are not affected by the data in the Database.

f you check the test results, you’ll see that the annotated methods are executed before and after the start of each test.

This manual way of cleaning up the state can help you deal with redundant tasks that need to be executed in each test, and it also reduces the length of your code, making it more readable.

It’s important to note that if you configure a fixture in your setUp, that test and the fixture can be combined and affect all of your tests.

That’s mean modifying the setUp shouldn’t affect other tests.

For this reason, if you have fixtures that may be executed in multiple test codes, you may want to extract them as separate factory methods to reduce coupling between tests.

@Transactional

Spring provides a very handy feature to automatically roll back if you specify the @Transactinoal annotation in your test code.

Let’s check the results of a test run without @BeforeEach and @AfterEach.

First test succeeds, but the second test doesn’t roll back the first test, so the result of the previous test, 3, is carried over and we see that there are 5 instead of the expected 2.

But what if we specified @Transactional in the test method?

As expected, the tests are successful because they automatically roll back when each test performs as expected.

While this provides the convenience of automatically rolling back with a single annotation, there are side effects such as unintended transactions being applied during test execution due to issues such as only one transaction boundary being used.

The side effects of using @Transactional in your test code are too extensive to cover in this article, so i’ll treat soon.

@DataJpaTest

The @DataJpaTest is an annotation that focuses on JPA components.

in spring docs

tests annotated with @DataJpaTest are transactional and rolled back after each test.
uses an in-memory db (replaces the explicit or normally auto-configured DataSource).
you can override these settings using the @AutoConfigureTestDatabase annotation.
SQL queries are logged by default by setting the spring.jpa.show-sql property to true, which can be disabled using the showSql property.
if you want to load the full apllcation configuration but use an embedded db, you should consider @SpringBootTest combined with @AutoConfigureTestDatabase rather than this annotation.

but let’s take a look at just what we need.

Set the logging level to DEBUG in application.yml and run the above test, you will see that

In fact, you can see that each test is rolled back after it is executed, and the tests are rolled back and succeed as intended, because we include the @Transactional annotation inside the @DataJpaTest annotation.

Quick and easy testing, focus on data accesses, etc. It’s a great annotation, but I’m wondering how it’s used in production testing and how I can use it to test more efficiently.

@SQL

The @SQL is an annotation that allows you to run SQL scripts before and after the test execution.

Scripts written in the *.sql file can be used to initialize or clean up the state of the DB to make it suitable for testing.

First, create a script file.

// truncate.sql
TRUNCATE TABLE product;

// ddl.sql
INSERT INTO product (id, name, quantity) VALUES (1, '테스트 상품1', 10);
INSERT INTO product (id, name, quantity) VALUES (2, '테스트 상품2', 20);

In this case, the location of the script file is under resources.

the first script (truncate.sql) puts the database in an initial state, typically including a command like TRUNCATE TABLE product; that deletes all the data in the table,
the second script (ddl.sql) inserted the initial data needed for the test or set up any necessary schema changes.

test is succeed.

Similar to @Before, AfterEach in Junit, the executionPhase property gives you the flexibility to control when the SQL script is executed.

You can set BEFORE_TEST_METHOD or AFTER_TEST_METHOD to run the script before or after each test method for flexible execution control.

As I mentioned above, testing with the @Transactional annotation has a lot of side effects, and the @SQL approach is an alternative.

@DirtiesContext

@DirtiesContext annotation indicates that the underlying Spring ApplicationContext became dirty (modified or corrupted in some way, such as changing the state of a singleton bean) during the test run and should be closed.

If tests using the same Application Context recycle an existing context rather than creating a new context for each test, the next test that runs might fail because the previous test changed the property value of a particular bean, or removed it, or changed the state of an object.

However, if we specify the annotation, a new context is created for each test run, and the underlying Spring container is rebuilt for all subsequent tests.

This is the first time I’ve learned about annotations, and I think it’s overkill to ensure independence by reloading the context every time you run a test, even though one of the principles of testing is to be fast.

Especially if the production code becomes huge, the overhead will be even bigger, so I think it’s better to look for other good alternatives.

Closing

Test code is just as important as production code.

Tests related to DB are especially tricky because the data changes every time you run the test, so you can’t guarantee consistent results, but I think you can write good tests if you understand the benefits and caveats of different methods.

I’m sure there are a lot of methods I didn’t cover, and I’ll have to find a good way to apply them in new projects.

Thanks for reading, and if you point out any incorrect information, I’ll fix it.