React: white-box vs black-box testing

Recently, a colleague and I were working together resolving an issue with a test for a React component. While working on this, we talked about two different ways of testing React components.

The first technique is how our component was tested initially. It was shallow rendered using Enzyme. Tests were asserting the internal state and properties of the component were set correctly. This is what I will refer to as white-box testing throughout this post.

The second technique is how we ended up rewriting the tests. Using the React Testing Library. RTL does not provide access to a component’s internals. Instead, you’re forced to work with the public API of the component (the props it accepts and the DOM it outputs). I’ll refer to this technique as black-box testing.

Black-box vs white-box testing

Let us take a look at the differences between black-box vs white-box testing.

Black-box testing

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings.

Source: Wikipedia.

White-box testing

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing).

Source: Wikipedia.

So where white-box is about testing the internals of a component, black-box is about testing the public facing part (the API) of a component.

Let’s see how the two approaches relate, and how they look like in code.

Writing tests using Enzyme

Enzyme has been around for a while now (the first commit was in 2015). It comes with helpers for making assertions while testing React components.

Enzyme helpers

• .props() give you the internal properties of a component.
• .setState() sets a component’s internal state properties.
• .instance() returns the underlying class instance of a component.

All of these helpers give you access to the internal’s of a component. The problem is that they encourage white box testing.

Example of white box testing

In the test below, we verify if the TopPosts component contains the three links to the top posts of a blog. We’re using Enzyme for the tests and React Router to render the actual links.

TopPosts.test.js

import React from 'react'
import { Link } from 'react-router-dom'
import { shallow } from 'enzyme'

import TopPosts from './index'

describe('TopPosts', () => {
  it('should fetch the top-posts and store them as state', () => {
    const dummyFetch = () => ['postA', 'postB', 'postC']
    const wrapper = shallow(
      <TopPosts fetchPostsHandler={dummyFetch} />
    )
    expect(wrapper.state('posts').length).toEqual(3)
  })

  it('should show the links to the top-3 posts', () => {
    const dummyFetch = () => ['postA', 'postB', 'postC']
    const wrapper = shallow(
      <TopPosts fetchPostsHandler={dummyFetch} />
    )
    const links = wrapper
      .find('li')
      .map(node => node.find(Link).prop('to'))

    expect(links[0]).toEqual('/post/postA')
    expect(links[1]).toEqual('/post/postB')
    expect(links[2]).toEqual('/post/postC')
  })
})

We render TopPosts and make sure the result of the fetchPostsHandler is stored as internal state. Next ,we lookup the Link components and get the to property of every link.

Finally, we assert that the links are rendered in a certain order. And that the to property matches the links we’ve passed.

What’s wrong with this test? We’re relying on how the component works internally. That can be fine for now, but it does come with some tradeoffs.

Risks when relying on the internals of a component

1. You might refactor TopPosts one day, and decide to pass the posts via props. We would remove fetching the posts, not store the posts internally anymore, but directly iterate the incoming data. The functionality of the component wouldn’t have changed (it still renders the top posts), but the test would fail.

2. If someday you replace the router library with something else, the test will fail too. The routing library might not expose Link with a to property, but instead a different API.
   Or what if you decide you don’t need a routing library anymore and just want to render anchor elements instead? The functionality of NavBar would be exactly the same, but our tests would fail. It’s fragile and sensitive to change.

Let’s see how we can improve on this!

Black-box testing

The test below is written using the React Testing Library (RTL). It does not give us access to the internals of the component, so we have to test its public API.

TopPosts.test.js

import React from 'react'
import { render } from '@testing-library/react'

import TopPosts from './index'

describe('TopPosts', () => {
  it.skip('should fetch the top-posts and store them as state', () => {
    // test is disabled, as we don't have access to the internal state anymore
  })

  it('should show the links to the top-3 posts', () => {
    const { container } = render(
        <TopPosts fetchPostsHandler={() => ['postA', 'postB', 'postC']} />
      )
    
    const links = container
      .querySelectorAll('a')
      .map(node => node.getAttribute('href'))

    expect(links[0]).toEqual('/post/postA')
    expect(links[1]).toEqual('/post/postB')
    expect(links[2]).toEqual('/post/postC')
  })
})

We still render the TopPosts component and pass the handler to fetch the top three posts.

The difference is in the API we use for our assertions. We use the API of the DOM. The DOM is the output this component exposes to the public and not something used internally.

Advantages of black-box testing

Less fragile tests

1. We can easily swap out our routing library. It would not impact our test. As long as our links are rendered correctly.
2. We don’t care about how TopPosts handles the posts. It can store them as state, in a local variable or any other way. We don’t care about the internals. As long as the links to our 3 top posts are being rendered.
3. Whatever library we now use to render the links, our assertions will still be valid. We expect three anchor elements linking to the URL of each post. We don’t rely on the NavBar using React Router’s Link component. And we certainly don’t assert if Link has a to property containing our link.

Improved design

1. It enforces us to think about the public API of a component. If I’m not able to treat it as a black box, how can I improve the design of the code so that we actually can.

Conclusion

Above we looked at two different ways to approach testing a React component. Using React Testing Library we don’t have access to helpers which give access to the internals of a component. And that limitation forces us to rethink the design of a component. It enforces good testing practices.

Kent Beck briefly mentions the issue with white-box testing it in his book Test-Driven Development by Example.

Wishing for white box testing is not a testing problem, it is a design problem. Anytime I want to use a variable as a way of checking to see whether code ran correctly or not, I have an opportunity to improve the design.

There’s a lot more to learn about this. Kent C. Dodds (the author of RTL) regularly writes about testing React applications on his blog.

• react
• javascript
• testing