February 1, 2022

Making Cypress Integration Tests Less Flaky


TL;DR

  1. Interleave Cypress commands like .find, .get, .first, .eq, .type with Cypress assertions like .should, .contains.

    Cypress runs only the last command when retrying. Interleaving act as guards to ensure we reach to the correct element which also helps avoiding detached parent errors.

  2. Don’t just wait for network calls, wait for the UI to be updated with the network data.

    If a network call has finished, it doesn’t mean the UI would be updated immediately.


Example 1

BAD:

cy.get('.parent').find('.child');

// If `.parent` got detached and rerendered,
// then the `.find` command would fail, as it
// would try to run `.find` on the now detached parent.
// This could happen in case of a loading `shimmer` component.

GOOD:

cy.get('.parent .child');

// While `.child` has not rendered, Cypress will
// keep retrying the last command, i.e. the complete
// cy.get command.
// Even if `.parent` got detached, Cypress would
// run the complete command when it is trying to
// find `.child`.

or

cy.get('.parent')
  .should('have.length', 3) // we made sure that the child has rendered
  .find('.child');

or

cy.get('.parent')
  .contains('some text after which i am sure a child has rendered')
  .find('.child')

Example 2

BAD:

cy.get('.new-todo').type('todo A{enter}');
cy.get('.todo-list')
  .first()
  .should('contain', 'todo A');

cy.get('.new-todo').type('todo B{enter}');
cy.get('.todo-list') // talking about this below
  .first()
  .should('contain', 'todo B');

// It might take some time for `todo B` to be visible on UI after
// `.type('todo B{enter}') because of async rendering/server calls etc.
// `cy.get('.todo-list')` in this case returns the list with only
// `todo A` because `todo B` is still not in UI. And then calling
// `.first()` returns `todo A`.
//
// Cypress retries only the last command which in this case would be
// `.first()` which would still return `todo A`, because
// `cy.get('.todo-list')` still returns old list.

or

cy.get('.new-todo').type('todo A{enter}');
cy.get('.todo-list:first')
  .should('contain', 'todo A');

cy.get('.new-todo').type('todo B{enter}');
cy.get('.todo-list:first')
  .should('contain', 'todo B');

// This would retry `cy.get('.todo-list:first')` which gets the
// correct UI element after retrying.
// This test is not flaky, but it uses `:first` which is deprecated in
// jquery and will be removed in jquery 4

GOOD:

cy.get('.new-todo').type('todo A{enter}');
cy.get('.todo-list')                        // command
  .should('have.length', 1)                 // assertion
  .first()                                  // command
  .should('contain', 'todo A');             // assertion

cy.get('.new-todo').type('todo B{enter}');
cy.get('.todo-list')                        // command
  .should('have.length', 2)                 // assertion
  .first()                                  // command
  .should('contain', 'todo B');             // assertion

// We have interspersed commands with assertions.

Example 3

BAD:

cy.wait('@myNetworkCall');

cy.get('.ag-center-cols-viewport').
  .find('.ag-row')
  .first()
  .click();

// The parent `.ag-center-cols-viewport` might get detached before
// finding the child

or

cy.wait('@myNetworkCall');

cy.get('.ag-center-cols-viewport .ag-row:first')
  .click();

// we might click on the first loading row of the table

GOOD:

cy.wait('@myNetworkCall');

cy.get('.ag-center-cols-viewpost')
  .contains('My data from network call now in UI')
  .get('.ag-row') // now sure we'll get the correct element because
  .first()        // of the assertion made above
  .click();

// check if UI state is updated in the parent container before querying
// for child element

Example 4

BAD:

cy.get('.ag-center-cols-viewport')
  .find('.ag-row')
  .first();
// element from `cy.get` or element from `.find` might get detached

or

cy.get('.ag-center-cols-viewport .ag-row')
  .first();
// element from `cy.get` might get detached

or

cy.get('.ag-center-cols-viewport .ag-row:first');

// :first is not a valid css selector. It is only supported by jquery, 
// but it is deprecated and will be removed in future.

GOOD:

cy.get('.ag-center-cols-viewport')
  .should('have.length', 10) // verify that we indeed have multiple rows
  .first();

or

cy.get('.ag-center-cols-viewport')
  .contains('data from network call') // verify data populated
  .first();

References

September 6, 2021

Optimizing Netlify

We’ll optimize Netlify’s Single Page web application load time.

Log in to https://netlify.com.

You’ll be redirected to https://app.netlify.com after logging in. This is the SPA we’ll be optimizing.

Open Chrome DevTools (cmd + options + i)
Select Performance Panel

Make sure Screenshot option selected (useful to check when app was loaded)

Start recording and refresh the page. Stop the recording when the page has loaded. We have the DevTools open in detached mode to view the timeline.

On closer look in the network section, it looks like the network call api.netlify.com/api/v1/user is duplicated. api.segment.io/v1/p is also duplicated but that doesn’t look much interesting.

We go to the Network panel of DevTools to check the details about this user api.

Now we check the call stack for both these calls.

Both call stack look pretty similar with one difference.

- App.js:432
+ App.js:459

Different lines in the same file:

We’re lucky Netlify has source-maps enabled in public, otherwise we’d see minified code.

The first useEffect is meant to run when the app loads for first time, at this time userId is not defined.

The second useEffect is running again when userId is not defined. It should be:

useEffect(() => {
  if (userId) {
    load(props);
  }
}, [userId]);

This will fix the api call being made twice.

Now back to the timeline, I see an opportunity for improving the app load time.

Looks like the main thread is not doing much while the network calls are being made. The current process is in series: the JavaScript runs and this code makes some network calls. We can do these in parallel because the network calls are handled by browser in a separate thread.

To do this we’d normally need the source to build the code but we’ll be using Chrome Local Overrides.

I have the main html file overview and main js file app.bundle.js overridden with my local copy.

I found a place where I’ll short-circuit the api call for user:

Updating this to

  user() {
    return window.userPromise || this.request('/user');
  }

Now we’ll define window.userPromise in the main HTML file because we want this api call made ASAP. We’ll add a new <script> tag and add our /user api call with the correct access token from the local storage.

And it works, we now have an api call at the start of page, in parallel as the main JavaScript code runs.

But there are 2 more network calls which are blocking the app render, let’s optimize them in the same way.

We now have a busy main thread, networks calls and JavaScript code are being run in parallel.

For my system and network, I could see around 40% reduction in app load time from 2000ms to 1200ms.

This is a common scenario in SPA using bundling systems like Webpack, API calls are made after the code is run. Early API calls is a simple method to improve app load time for a lot of web apps.