
I recently released a new version of [logd](https://logd.co), a side project where part of the vision is to enable just starting to type notes without the ceremony of creating a new document, etc.

In that spirit, I made it possible for anonymous users to create notes that are saved locally, making it a kind of ad-hoc scratch pad. You can [try that out now](https://logd.co) if you'd like.

However, I also wanted to provide the option to save those local notes to the cloud on login.

I did quite a bit of googling around for a solution that would support this but did not find any patterns I liked. One reason might be that this is a somewhat non-standard user flow. In a conventional model, you sign in first, and then create a note.

Since others might be interested in supporting saving anonymous content to a database, I decided to write a blog post about how I implemented this feature. 

## Solution overview
For this solution, I decided to use a combination of localStorage and query params to manage the process of saving anonymous content to a database. (At the end, I'll talk a bit about some other option I considered.)

Here is an overview of the solution, which I'll walk through in detail below:

1. **Persist changes locally:** While a user is anonymous, their note content is persisted via local storage. Nothing unusual there.
2. **Use a param to trigger saving to the database:** When a user clicks login or signup, we check to see if any local content has been entered, and if so append a query param with the "save anon content" route where we'll handle saving to the database.  We reuse the same query param we'd use if a user tried to access a restricted page.
3. **After login, redirect to the "save anon content" page:** After login completes, our redirect param will send the user to the "save anon content" page.
4. **Save local content to the database:** On the "save anon content" page, we retrieve the local content, save it to the database, which generates an id, which we then use to redirect the user to the newly created note page.
4. **Redirect to our new note and delete the local content:** Finally, after redirecting to the new note page, we also include a param that will enable deleting the localStorage content, since it is now safely in the database.

## Prerequisites and Setup 
I implemented this version of Logd notes using the [Remix](https://remix.run/) framework with Node, and I'll therefore also use the same stack for this walk-through.  I'm also assuming you already have basic knowledge of React with TypeScript.

You can find a fully working solution in [this repo](https://github.com/andersr/save-local-to-db-example).

I'll be using the [Remix Indie Stack](https://github.com/remix-run/indie-stack) as a starting point since it already includes both authentication as well as notes CRUD functionality. If you're following along writing code, you'll want to first create a new app using the Remix Indie Stack, as follows: `npx create-remix@latest --template remix-run/indie-stack`. 

Let's get started!

##  1. Persist changes locally
This part is mostly basic functionality for persisting changes locally, with an added simple check for if content has been entered, and a function for appending a query param to the login link.  Open up `app/routes/_index.tsx` and make the following updates:

```tsx
// app/routes/_index.tsx
...
export default function Index() {
...
  // hook for persisting changes locally
  const [value, setValue] = useLocalStorage(
    ANON_USER_LOCAL_STORAGE_CONTENT,
    "",
  );

  // simple check for if content has been entered
  const noteHasContent = (value as string).trim() !== "";

  // display current save status
  function displaySaveStatus() {
    if (noteHasContent) {
      return (
        <span>
          Saved locally.{" "}
          <Link to={`/login${setParam()}`} className="text-gray-400 underline">
            Save to my notes
          </Link>
          .
        </span>
      );
    }
    return "";
  }

  // set the param if content has been entered
  function setParam() {
    return noteHasContent
      ? `?${REDIRECT_TO_PARAM}=${encodeURIComponent(SAVE_ANON_ROUTE)}`
      : "";
  }

  // replace the current view with the following
  return (
    <div className="flex h-full min-h-screen flex-col">
      <header className="flex items-center justify-between bg-slate-800 p-4 text-white">
        <h1 className="text-3xl font-bold">Anon Note To Db Example</h1>
        {user ? (
          <div>
            <Link to="/notes" className="">
              View Notes for {user.email}
            </Link>
          </div>
        ) : (
          <div className="">
            <Link
              to={`/join${setParam()}`}
              className="rounded bg-slate-600 px-4 py-2 text-blue-100 hover:bg-blue-500 active:bg-blue-600 mr-4"
            >
              Sign up
            </Link>
            <Link
              to={`/login${setParam()}`}
              className="rounded bg-slate-600 px-4 py-2 text-blue-100 hover:bg-blue-500 active:bg-blue-600"
            >
              Log In
            </Link>
          </div>
        )}
      </header>
      <main className="p-8">
        <div className="w-[400px] mx-auto">
          <textarea
            className="w-full rounded-md border-2 border-blue-500 px-3 py-2 text-lg leading-6"
            value={value}
            rows={8}
            onChange={(e) => setValue(e.target.value)}
            placeholder="Write something..."
          />
          <div className="py-2 text-sm text-slate-500">
            {displaySaveStatus()}
          </div>
        </div>
      </main>
    </div>
  );
}
```

With the above update, a redirect param will be passed when clicking signup or login, which we'll make use of in the next step.

##  2. Use a param to trigger saving to the database
Now, when the user clicks on login or signup, if they've typed in any content, a param will be appended to the URL. Note the use of `REDIRECT_TO_PARAM` and `SAVE_ANON_ROUTE`.  I prefer to store anything pases as a param as a constant since it will be used in at least two places (the originating link and the handler on the target page), so we want to ensure they match.

Also, note the use of [encodeURIComponent](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent), which ensures passed values are properly formatted for appearing in a URL.

## 3. After login, redirect to the "save anon content" page
On the login page, we shouldn't need to make any updates, since we're re-using the existing [`redirectTo` functionality](https://github.com/andersr/save-local-to-db-example/blob/d60553d276ed52ffc24b35b0432602de49863804/app/routes/login.tsx#L20). After login, the app should redirect to the route we passed in (`SAVE_ANON_ROUTE`). Note that the actual route can be whatever you want. In my case it's `/save-anon-note`.

### Why a separate page?
Let's talk a bit about why we have a separate view just for saving local content to a database.  One fair question is why we can't just complete this task right on the login page after the user signs in? However, that would not be a good idea, since, once the user is authenticated, they need to be redirected _away_ from the login page, as that page should only be accessible to anonymous users.

Ok, so why not just handle everything on the server after the user is signed in?  Unfortunately, that will not work, because localStorage is, by definition, [only accessible on the client](https://developer.school/snippets/react/localstorage-is-not-defined-nextjs).  We therefore need an intermediate page that we fully load so that we can retrieve the local content on the client and then save it to the database on the server.

### 4. Save local content to the database
Thanks to the param we passed on the homepage, we are redirected to the "save anon" page. This view contains the core part of the solution.

Go ahead and create a new page at `app/routes/save-anon-note.tsx` and add the code below.

```tsx
// app/routes/save-anon-note.tsx
import type { ActionArgs, LoaderFunction } from "@remix-run/node";
import { json, redirect } from "@remix-run/node";
import { useNavigate, useSubmit } from "@remix-run/react";
import { useEffect } from "react";
import { LiaSpinnerSolid } from "react-icons/lia";
import { createNote } from "~/models/note.server";
import { getUserId, requireUserId } from "~/session.server";
import {
  ANON_USER_LOCAL_STORAGE_CONTENT,
  LOCAL_NOTE_SAVED_PARAM,
} from "~/shared";

// redirect users who are not signed away from this page
export const loader: LoaderFunction = async ({ request }) => {
  const userId = await getUserId(request);
  if (!userId) return redirect("/");
  return json({});
};

// handle the programmatic form submit
export const action = async ({ request }: ActionArgs) => {
  const userId = await requireUserId(request);

  const formData = await request.formData();
  const title = formData.get("title");
  const body = formData.get("body")?.toString() || "";

  if (typeof title !== "string" || title.length === 0) {
    throw redirect("/", 400);
  }

  const note = await createNote({ body, title, userId });

  return redirect(`/notes/${note.id}?${LOCAL_NOTE_SAVED_PARAM}=true`);
};

export default function SaveAnonNote() {
  const navigate = useNavigate();
  const submit = useSubmit();

  useEffect(() => {
    // after the page has mounted, look for content in local storage
    const localContent = window.localStorage.getItem(
      ANON_USER_LOCAL_STORAGE_CONTENT,
    );

    if (localContent) {
      handleAnonNote(localContent);
    } else {
      console.warn("no local content found");
      navigate("/");
    }

    function handleAnonNote(localContent: string) {
      try {
        const noteContent = JSON.parse(localContent);
        if (noteContent.trim() === "") {
          throw new Error("No note content");
        }

        const lines = noteContent.split("\n");
        const title = lines[0];
        const body = lines.length > 0 ? lines.slice(1).join(" ") : "";

        let formData = new FormData();
        formData.append("title", title);
        formData.append("body", body);

        submit(formData, { method: "post" });
      } catch (error) {
        console.warn("anon note error: ", error);
        navigate("/");
      }
    }

  }, [navigate, submit]);

  // on the page itself, we're just displaying a spinner while this brief process completes
  return (
    <div className="flex h-full flex-col items-center justify-center">
      <LiaSpinnerSolid
        size={"30px"}
        title="Loading"
        className="text-primary animate-spin text-6xl"
      />
    </div>
  );
}
```

There's quite a bit going on here, so let's break it down step-by-step:
- On the server side (in the `loader` method), we confirm the user is authenticated and redirect away if they are not.
- We use the `useEffect` method to determine that the page has mounted and that we are able to read from localStorage.
- If no local content is found, we redirect to the homepage, canceling the process.
- If local content is found, we do some basic validation and then parse out the content title vs body. (This part is not really specific to saving anonymous content to a database.) Then, we use Remix's [useSubmit](https://remix.run/docs/en/1.19.3/hooks/use-submit) hook to programmatically submit our note values to the server, after which point we handle the process very similarly to as if a user had created a new note.
- See the `action` method for how we handle the form submit, create the note and redirect to the newly created note. Note also that we are passing in a `LOCAL_NOTE_SAVED_PARAM` which we'll use to remove the local content after the redirect.
- Finally, on the page itelf, we just display a spinner while we complete the above process.

### 5.Redirect to our new note and delete the local content
If all goes well, we redirect to the new note and pass our `LOCAL_NOTE_SAVED_PARAM` param.
On the note detail view, we check for this param and use it to safely clear local storage, knowing that it has been saved to the database. And with that we are done <Emoji symbol="🎉" label="celebrate emoji" />.

```tsx
// Only relevant snippets included
// See app/routes/notes.$noteId.tsx for the complete code
...
import {
  ANON_USER_LOCAL_STORAGE_CONTENT,
  LOCAL_NOTE_SAVED_PARAM,
} from "~/shared";
...
export default function NoteDetailsPage() {
  ...
  const navigate = useNavigate();
  const [searchParams] = useSearchParams();
  const localNoteSaved = searchParams.get(LOCAL_NOTE_SAVED_PARAM);

  useEffect(() => {
    // if the param is found, remove the local content and redirect to the current page, and remove the url that had the param from this history stack with replace set to true.
    if (localNoteSaved) {
      window.localStorage.removeItem(ANON_USER_LOCAL_STORAGE_CONTENT);
      navigate(location.pathname, { replace: true });
    }
  }, [localNoteSaved, location.pathname, navigate]);
...
```

## Final Thoughts
I considered a few other solutions before going wih the one described above.

For example, I considered creating a guest session for every anonymous user.  This would have the advantage of allowing for just storing everything in the database.  However, it also comes with some security implications and also felt a bit more complex.  I also considered using a JWT stored as cookie as a temporary note id.

But in the end, I felt the above solution, while containing many small steps, was the simplest and fastest to implement.  Additionally, there are fewer security concerns, since I do not write to the database until the user has been authenticated.  And last but not least, I am using query params to manage the flow, which is a tried and true pattern.

This is not a very complex solution, but it does involve a series of small steps, which means there is quite a bit of surface area for failure. For this reason, it's a great candidate for one or more E2E tests, something I might write about in another post.



For my new notes app, I wanted to allow anonymous users to create notes locally, and then be able to save it to a database on login.  In this blog post, I walk through how I implemented this.

How to save anonymous content to a database


If you create a new directory in your terminal, what is your likely next step?  Probably cd'ing into the directory.

In other words, you're first doing `mkdir somedir` followed by `cd somedir`. 

If I find myself typing something like this on a regular basis, I'll create a custom command for it.

## Create a Directory and `cd` Into It In One Command

Here is the very simple function `mkcd` I created. 

```bash
function mkcd
  mkdir -p $argv ; cd $argv 
end
```

Now all you need to do is type `mkcd` and you've combined the two steps into one.

The above is written for the [fish shell](https://fishshell.com/).  Adding custom commands in Fish is ridiculously trivial.  Here is [a great article](https://medium.com/@yowats0n/creating-a-global-function-in-fish-shell-238b571dc6df) describing the process.

If you use a bash shell, your script will look something like this:

```bash
function mkcd() {
  mkdir -p $1 ; cd $1
}
```

And here you can learn about [adding custom commands in bash](https://shanelonergan.github.io/streamline-your-workflow-with-custom-bash-commands/).

Here are a few more custom commands I use quite a bit.  They're written for Fish, but should be easy to rewrite for bash.

## New Git Branch
Checking out a new branch with the `-b` modifier in order to create a new branch is not intuitive and a lot to type. I decided I prefer `gnew` instead. 

```bash
function gnew
  git checkout -b $argv
end
```


## Delete git branches
Now that we've added branches, we want an easy way to delete them.

```bash
function gbd
  git branch -d $argv
end
```
Now you can just type `gbd first-branch other_branch  FOObranch` and git will attempt to delete them all. I use the lowercase `-d` so Git will tell me if the branch is not fully merged.

## NPM Uninstall
Much faster than typing `npm uninstall`.


```bash
function nu
  npm uninstall $argv
end
```


## Just `g` for git
Since Git is something I use all day, I created what basically is an alias for `git`.

```bash
function g
  git $argv
end
```

Now I can just type `g push` or whatever.  I've done the same for other very common commands I type, eg `y` for `yarn` and so forth.

## Git First Push
I like to be able to just type `g push` when I am in a branch, to push new code.  However, for that to work I need to tell Git that I want to push to the remote branch matching the name of the branch I am in.  Here is a script for doing that.

```bash
function gfpush
  git push --set-upstream origin (git status | head -1 | string split ' ')[-1]  $argv
end
```

Basically, I use this command the first time I push code in a new branch, and after that I can just type `g push`.

Those are a few of my favorites. 


## My one tip for creating custom commands
As a closing thought, if you are considering writing your own custom commands, don't do what I did initially, which was to combine way too many things into one command. <Emoji symbol="😭" label="crying emoji" /> 

Not surprisingly, the custom commands I use the most are the ones that are simple and do one, or maybe two, things.

Some of My Favorite Shell Commands


One of my pet peeves when it comes to front-end development is seeing a footer floating in the middle of the page.  This usually happens because whoever built the page just placed the footer structurally below the main content and then I guess assumed there would be enough content to push the footer to the bottom of the page.  Or maybe some other reason.  Either way, the end result is that pages with very little content, or when viewed in a tall viewport, look kind of wonky. 

Here are a couple (anonymized) examples:

![Example of page with a footer in the middle of the screen.](/images/blog/floating-footer-example-2.jpg)

![Example of page with a footer in the middle of the screen.](/images/blog/floating-footer-example.jpg)

These are just from the last few days. I see these all the time, and sometimes on fairly major sites. <Emoji symbol="😭" label="crying emoji" /> 

If either of the above (or countless others I see regularly) were a website I was working on, especially if I were the front-end dev who built this page, I'd be pretty embarassed.

<GiphyEmbed src="https://giphy.com/embed/apowQdwqKcKPK" href="https://giphy.com/gifs/embarrassed-dog-apowQdwqKcKPK" />

Yes, it's a small thing.  But it's a small thing sort of like forgetting to zip your fly is a small thing.  <Emoji symbol="😳" label="embarassed emoji" />  It's not a good look and kind of hard to ignore once you notice it. 

But the good news is it's easy to fix.  

This is probably one of the simplest ways to keep your footer at the bottom of your viewport or below it: 

```html 
<html>
    <head>
        <style>
            html {
                height: 100%;
            }

            body {
                margin: 0;
                min-height: 100%;
                display: flex;
                flex-direction: column;
            }

            main {
                flex: 1;
            }
        </style>
    </head>
    <body>
        <main>PAGE CONTENT</main>
        <footer> This will be at the bottom of the page, even if there is little or no page content.  If there is a lot of content, it will be pushed below the page fold.</footer>
    </body>
</html>
```

This will work nicely on most browsers, and if viewed on a very old browser (eg IE 10), it will just "gracefully degrade" to the old floating footer.

[Here is the above markup in a sample page](/examples/full-height-footer-at-bottom-no-content.html).

[Here is a version with lots of content](/examples/full-height-footer-below-viewport-with-content.html).  As you see, the page becomes scrollable with the footer now disappearing below the fold, as expected.



If this blog post results in just one less ugly floating footer, I will consider it a success :-)


Prevent the annoying "floating" footers in the middle of your screen.

How to Keep Your Footer at the Bottom of the Screen (or Below)


_This post is part of the [E2E Testing Series](/blog/e2e-testing-series).  If you haven't already, I suggest reading previous posts before continuing._ 

## Process Overview

We're going to walk through a sample process for how to write E2E tests and touch on some best practices along the way.  Here's an overview of the general process I tend to follow:

1. Create a test file and add pending tests for all the feature requirements.
2. Write the minimum code needed to get each test to pass.
3. With the test runner in watch mode, go back and refactor and clean up code as needed.

If you're familiar with [Test-Driven Development](https://en.wikipedia.org/wiki/Test-driven_development) (TDD), the above might look familiar.  I've found that writing E2E tests is a great opportunity to put TDD into practice.

As our sample feature, we'll be walking through writing the tests for the Newsletter signup form I discussed in the series intro.
 ![newsletter form](/images/blog/e2e-testing/newsletter-signup-form.jpg)
I picked it because it's a small but still complete feature, so a great basis for conveying the E2E test-writing process.

To get started, we'll create our main test file.


## Create the test file and add pending tests
If possible, I suggest collecting all tests for each feature in the same test file, and then match the name of file to the test title.  Let's do that now.

(I'm assuming you've already [set up your local environment](/blog/e2e-series-setting-up-your-local-environment).)

Create a new file at `cypress/e2e/newsletterSignup.cy.ts`.

Note that in [version 10 of Cypress](https://www.cypress.io/blog/2022/06/01/cypress-10-release/) E2E test files are placed in the directory `cypress/e2e` and use the `.cy` extension in the file name, which replaces the previous `.spec` extension for E2E tests.

Add the following code to this file:

```tsx
// cypress/e2e/newsletterSignup.cy.ts

describe("Newsletter signup", () => {  // ** 1 **
  beforeEach(() => {  // ** 2 **
    cy.visit("/newsletter");
  });
   
  it("displays a message explaining the form's purpose");  // ** 3 **

  it("allows for completing a newsletter signup");

  it("allows for submitting the form by using the enter key");

  it("displays an error message if input value is not a valid email address");

  it("displays an error message if the input field is empty");

  it("clears the email input if signup is successful");

  it("persists the email input value if there is an error");
});
```
### Code notes
1. This is the `describe` block, which is the method used to group test specs.  I recommend matching the name of this block (_e.g._ here the name is "Newsletter signup") to the name of the test file and, if possible, also to what this feature is named elsewhere, eg in a User Story. This is a convention I like to follow to encourage only having one set of feature tests per file and also to create less confusion when trying to locate a test.  The value of this will become more apparent when you start to have large numbers of tests.
2. The `beforeEach` block will run before every test in the describe block.  This is a great place to tell the test runner where it should navigate to.  Also, when applicable it can be a good place to place the app in desired state, eg using a login command. 
3. This is our list of [pending tests](https://glebbahmutov.com/blog/cypress-test-statuses/#pending-tests). The test descriptions for these tests should together make up the requirements for this feature.  If you are working off a User Story, the list of tests should match your [acceptance criteria](https://en.wikipedia.org/wiki/User_story#Acceptance_criteria). 

## Using your E2E tests as your "live" todo list
At this point, your Cypress console should look something like this:

<Image src="/images/blog/e2e-testing/e2e-testing-console-with-test-added.jpg" alt="E2E Testing console with newsletter test added" />

Go ahead and click on "newsletterSignup" and you should now see the test runner with a list of pending tests.

<Image src="/images/blog/e2e-testing/cypress-runner-with-pending-tests.jpg" alt="Cypress runner with the pending tests" />

With the test runner now in watch mode, as you update your tests and source code, our tests will re-run automatically and update tests to passing or failing.

I personally find it to be a great approach to have this type of "live" todo list, helping me remain assured that I am fully aligned and building what actually is being requested, and having a clear sense of my progress.

## Adding our first test 
We're going to start by getting the first pending test to pass, which also happens to be fairly basic. 

In `cypress/e2e/newsletterSignup.cy.ts`, go ahead and replace the first pending test with the following code:

```tsx
it("displays a message explaining the form's purpose", () => {
  cy.get('[data-test="newsletter"]').contains("Sign up for our newsletter");
});
```

Cypress tests allow for [chaining of methods](https://www.w3schools.com/jquery/jquery_chaining.asp), which means they often are one-liners like the one above.

The test divides into two parts:
- The [`cy.get`](https://docs.cypress.io/api/commands/get) part, which is where we select the element we want to test.
- The [`.contains`](https://docs.cypress.io/api/commands/contains) part which is one of many types of assertions.  In this case, we are asserting that we will find a specific text block in the previously selected element.

(At the risk of causing some confusion, `.get` is also an assertion that an element exists, and `.contains` can also be used to select an element, for use in other assertions.)

You might have noticed the use of `'[data-test="newsletter"]'` to identify our newsletter DOM element, which might look a bit odd.  We'll talk about that in more detail in a bit, but first let's get the above test to pass.

## Getting our first test to pass
If we look at our test runner, we see that the test we added above is failing.

<Image src="/images/blog/e2e-testing/first-failing-test.jpg" alt="E2E Testing console with first failing test" />

This is expected, since we haven't yet added the code to make it pass.  Let's do that now.

Add a new file at `src/pages/newsletter.tsx`.  Since we are placing a file in the special `pages` directory, NextJS will create the new corresponding route `/newsletter`.

Add the following code to the new file:

```tsx
// src/pages/newsletter.tsx
import type { NextPage } from "next";

const Newsletter: NextPage = () => {
  return (
    <form data-test="newsletter">
      <h3>Sign up for our newsletter!</h3>
    </form>
  );
};

export default Newsletter;
```

If you now go back to the test runner, you should see that there is a green checkmark next to your first test, which means it's passing!  <Emoji symbol="🎉" label="celebrate emoji" />

<Image src="/images/blog/e2e-testing/first-passing-test.jpg" alt="First passing test!" />

## Making tests more resilient with Test IDs 
Note the `data-test="newsletter"` attribute added to the `form` tag above. 

```tsx 
<form data-test="newsletter"> 
```

`"newsletter"` is our "test id" and what we use to select this element in our test with the selector `[data-test="newsletter"]`.

```tsx 
cy.get('[data-test="newsletter"]')...
```

This is a fairly common pattern in E2E testing, where we make use of a [custom data attribute](https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/data-*) to identify the element(s) we want to test.  Note that you can put whatever you want after the `data-` part, but I prefer using `test`.

Why would we want to do this?

I think the easiest way to understand the benefits of using test ids, is to look at the alternatives to identifying elements on a page:

- Use a DOM selector, such as maybe adding an ID to the element, eg `<form id="signupForm">`.
- Using a class name, eg `<form class="signup-form">`.
- Create a selector based on the current DOM structure, such as `main form h3`.

The main issue with these approaches is that they're all fragile.

Every time a developer touches this code, it is possible they might modify any of these due to some implementation need, and unknowingly cause a false negative test failure.

Use of test ID allows us to separate test-specific code and implementation code, making tests more resilient to unrelated code changes.

## Adding the main user flow test 
Next, let's add a test for the main user flow: completing and submitting the signup form.

This test will be a bit more advanced.  Go ahead and update the next pending test to be as follows:

```tsx
// cypress/e2e/newsletterSignup.cy.ts
it("allows for completing a newsletter signup", () => {
  const testEmail = "test@email.com";

  cy.intercept("POST", "/api/newsletter", (req) => { // ** 1 **

    expect(req.body.email).to.contain(testEmail); // ** 2 **

    req.reply({  // ** 3 **
      statusCode: 200,
      body: {
        success: true,
        message: "Success message",
      },
    });
  });

  cy.get('[data-test="emailInput"]').type(testEmail); // ** 4 **
  cy.get('[data-test="formSubmit"]').click(); // ** 5 **
  cy.get('[data-test="successMessage"]').should("exist"); // ** 6 **
});
```

The goal of this test is to ensure that a "happy path" newsletter signup flow works as expected.

### Code Notes
There's quite a bit going on in this test. Let's walk through it step by step:

1. `cy.intercept("POST"...` - here, we are telling Cypress to [intercept](https://docs.cypress.io/api/commands/intercept) a network request that will happen later in the test. With the intercept feature, Cypress basically masquerades as the given endpoint we defined and returns the response we expect.  The intercept feature is truly powerful and I highly recommend reading more about it in their docs. In our case, it allows us to fully test our client side form by actually submitting an email address without having to worry about clogging up some database with fake test email addresses.
2. This is where we assert that the client side sent the payload we expected to the back-end.
3. If the previous assertion passes, we send the expected success response.
2. `cy.get('[data-test="emailInput...` - this is where we type in the test email address.
3. `cy.get('[data-test="formSubmit"]').click();` - here, we click the form submit button and, which results in an API call, that then is intercepted as per step 1 above.
4. `cy.get('[data-test="successMessage"]')...` - assuming our form submit was successful, we expect a success response from the server.

## Getting the main user flow test to pass

Let's add the code needed to get the above test to pass. Update `src/pages/newsletter.tsx` with the following:

```tsx
import type { NextPage } from "next";
import { useState } from "react";
import { ServerResponse } from "../pages/api/newsletter";

const Newsletter: NextPage = () => {
  const [emailInput, setEmailInput] = useState("");
  const [confirmMessage, setConfirmMessage] = useState("");

  const handleSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
    setConfirmMessage("");
    e.preventDefault();

    const raw = await fetch("/api/newsletter", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        email: emailInput,
      }),
    });

    const response: ServerResponse = await raw.json();

    if (response.success) {
      setConfirmMessage(response.message);
      setEmailInput("");
      return;
    }
  };

  return (
     <form data-test="newsletter" onSubmit={handleSubmit}>
        {confirmMessage && (
          <div data-test="successMessage">{confirmMessage}</div>
        )}
        <h3>Sign up for our newsletter!</h3>
        <input
          type={"email"}
          value={emailInput}
          onChange={(e) => setEmailInput(e.target.value)}
          data-test="emailInput"
          id="emailInput"
          formNoValidate
        />
        <input type={"submit"} data-test="formSubmit" />
    </form>
  );
};

export default Newsletter;
```

We also need to add an API endpoint for submitting our form data.  Create a new file at `src/pages/api/newsletter.ts` and add the following:

```tsx 
import type { NextApiRequest, NextApiResponse } from "next";

export type ServerResponse = {
  message: string;
  success: boolean;
};

export default function handler(
  req: NextApiRequest,
  res: NextApiResponse<ServerResponse>
) {
  const { email } = req.body;

  res.status(200).json({
    message: \`\"\$\{email\}\" was added to the mailing list. Thanks for signing up!\`,
    success: true,
  });
}
```

(I won't be going through this code in detail, as this is not a post about TS or React.)

Note that this API endpoint doesn't actually store the submitted email in a data store. However, if this were a real newsletter signup, the place to do it would likely be in the above file.

If we now check our test runner, we should see that the main user flow test ("allows for completing a newsletter signup") test is passing.

<Image src="/images/blog/e2e-testing/main-user-flow-passing.jpg" alt="Main user flow test passing." /> 
 
## Getting remaining tests to pass
We've written the minimum code needed to get the current tests to pass. However, there are still several requirements that need to be fulfilled, such as ensuring that emails are validly formed. You'll find the completed tests and the corresponding code to get them to pass in the [github repo](https://github.com/andersr/e2e-testing-tutorial).

## Refactor and clean up your code with confidence
Now that we have passing tests for all the feature requirements, we can reap the benefits of our efforts when refactoring.

For example, we may want to DRY up our code, which, at least in my experience, can often lead to unintentionally breaking things.  Additionally, I haven't added much in the way to styling, and that is also something we now can do with far less concern about breaking something.

## Next Up: Ensuring tests run automatically on every code push
Now that we've written some tests, let's be sure and put them to work.   In the next post, we'll show how to make sure your full test suite runs every time you push new code, and why that matters. [Continue to the next post in the series](/blog/e2e-series-ensure-tests-run-on-every-push).

Part of the E2E testing series.  A lot of tutorials focus on configuring and running E2E tests. Here we focus on how to approach writing the actual tests.

A Suggested Process For Writing E2E Tests


_This post is part of the [E2E Testing Series](/blog/e2e-testing-series). If you haven't already, I suggest going back and reading previous posts before continuing._

## Writing tests is only the beginning

We've walked through the process of how to write E2E tests. We showed the benefits of writing tests even before you write any code. And we showed the value of having your E2E tests running in watch mode to get quick feedback.

While this is all great, it's unfortunately not enough. While we're doing a great job of writing our tests we're still not going to get quick feedback on a general app regression. (Remember, when you are running tests in watch mode, that is only covering the current feature you are working on, and not on the full test suite.)

To achieve that, we need to ensure that our full test suite runs when we add new code and that we are notified about any failures before we merge the code.

These are the three main steps we'll complete in this post to achieve that goal:

1. Enable running the full test suite against a production build.
2. Automatically run the full suite when we push new code.
3. Prevent merge of code into the main branch unless all tests pass.

There are a number of tools and techniques we can use to achieve the above. In our case, we're going to use a combination of npm scripts, [Github Actions](https://docs.github.com/en/actions), and [Github branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches).

Let's get started!

## Enable running the full test suite against a production build

Up until now, we've only been running the specific tests associated with the feature we are working on, and we've been running them against the development server. However, to prevent regression, we need to run all the tests, and run them against a production build. This is the version of your code that is served to users on the live site, and that is what we want to run our full test suite against.

### Add full test suite scripts

Most of what we need to do is to add the necessary scripts to run the full suite. Go ahead and make the following updates to `scripts` in your `package.json` file:

1. **Rename the `"build"` script to `"prestart"`.** This leverages a [scripts feature](https://docs.npmjs.com/cli/v8/using-npm/scripts) in npm, in which adding `pre` to another script will run that script first. In this case, we want to run a build before we run the `start` script, which is what is run to start the production server.
2. **Update the `start` script to use a different port:** Add ` -p 8080` to the end of the `"start"` script. This will allow for running the production server locally while also running the dev server, eg for debugging purposes. (Port 8080 is just my personal preference, you can pick a different port if you wish.)
3. **Add a script for running the full Cypress test suite:** `"cy:run": "cypress run --e2e --browser chrome --config baseUrl=http://localhost:8080"`. In contrast to the `"cy:open"` script, this will run the full test suite in [headless](https://www.softwaretestinghelp.com/headless-browser-testing/) mode, and we can use it both for local debugging and to run in a deployed environment. Note that the port used in `baseUrl` has to match what we use in the `start` script.
4. **Add a single command E2E script**: ` "e2e": "start-test start 8080 cy:run"`. This script pulls together all the other scripts above. We are first running the `start` script (which in turn will be preceded by the `prestart` build script), then waiting for a server to be listening on port 8080, and finally running the full test suite against that port. As before, this port has to match the two other ports you set previously.

Your `scripts` section should now include the following scripts:

```bash 
"scripts": {
     ...
    "prestart": "next build",
    "start": "next start -p 8080",
     ...
    "cy:run": "cypress run --e2e --browser chrome --config baseUrl=http://localhost:8080",
    "e2e": "start-test start 8080 cy:run"
  },
```

To make sure everything is working as expected, go ahead and run the `e2e` script, and you should see the entire above sequence being run, concluding with an 'All specs passed!' message in your terminal:

![Run all E2E tests locally with all tests passing](/images/blog/e2e-testing/e2e-run-all-tests-passing.jpg)

We'll be using this last script in the next section when pushing our code, but you can also use it locally for debugging purposes, such as when you are getting a test failure only when pushing code, but not seeing an error when running in watch mode.

## Automatically run the full suite when we push new code

As a reminder, this and the next step in this tutorial assumes you are using [github](https://github.com/) to store your code. There are many other options out there for achieving the same goal.

To ensure our `e2e` script runs on all new code that we push, we're going to add a [github action](https://docs.github.com/en/actions). Add the following to `.github/workflows/e2e.yml` (You must place the file in this directory, though the name `e2e` is my personal preference.)

```bash
name: E2E Tests

on: [push] # ** 1 **

jobs:
  cypress-run:
    runs-on: ubuntu-20.04 # ** 2 **
    steps: 
      - name: Checkout
      uses: actions/checkout@v2 - name: Cypress run
      uses: cypress-io/github-action@v2 # ** 3 **
    with:
      command: npm run e2e # ** 4 **
```

This code will be triggered when we push to a github repo.

### Code Notes

1. This is telling github to run this action every time we push new code.
2. This is the OS and version used for the virtual environment in which run our tests. I recommend always using a specific version, such as the latest stable version, rather than `latest`, which is risky because you may then suddenly start to see test failures caused by a version update that has nothing to do with your tests. See [virtual-enviroments](https://github.com/actions/virtual-environments) for the latest stable version.
3. We are using a ready-made [action](https://github.com/marketplace/actions/cypress-io) created by Cypress to run our tests.
4. This script name needs to match the `e2e` script we created earlier.

Go ahead and commit this code and [push it to a github repo](https://docs.github.com/en/get-started/importing-your-projects-to-github/importing-source-code-to-github/adding-locally-hosted-code-to-github).

Open up your repo in your browser and click on the ["Actions"](https://github.com/andersr/e2e-testing-tutorial/actions) tab, and you should see your action running.

![Github action running](/images/blog/e2e-testing/gh-e2e-action.jpg)

You can click into the action and see a detailed log of all the action steps.

## Prevent merge of code into the main branch unless all tests pass.

Just one more step to go!

Now that we've got our test running automatically on push, we want ensure that we don't merge code with failing tests. Similar to above, these instructions are specific to Github, but the general principles should be applicable regardless of the service you are using.

Achieving this requires setting up a branch protection rule.

### Protecting the main branch

The main branch (ie the branch containing code that will be pushed to production) is the one we need to protect from any code that contains regressions.

In github, we can add this protection by creating a branch rule, which should:

1. Not allow pushing code directly to the main branch, but instead require a pull request.
2. Require that all checks, in our case just the E2E tests, pass before merge is allowed.

You can find instructions for how to do add branch rules in the [github docs](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches), but very briefly, in your repo, click on Settings > Branches > Add Rule.

In "Branch name pattern", enter the name of your main branch, eg "main" and check the box "Require a pull request before merging". (I'll also uncheck "Require reviews" for the purposes of this tutorial, but it's probably a good idea to leave it checked.)

Next,check off "Require status check to pass before merging", which will prompt you to "search for status checks". Somewhat confusingly, a "status check" corresponds to the name of _job_ within an action rather than the name of the action itself. In our case, the name is `cypress-run` (which you'll see under `jobs:` in `.github/workflows/e2e.yml`.) Go ahead and type that in and you should see the job name to select.

Your form should look something like this:
![Adding main branch protection rule](/images/blog/e2e-testing/main-branch-protection-rule.jpg)

Scroll down and click on "Create".

## Testing that everything works

Let's make sure that our new branch rule works as expected. Go ahead and make a change to a test that you know will cause a failure, eg by adding an extra character to a test id.

To confirm that your test will fail, run your test suite locally using the `e2e` script.

Next, go ahead and commit the change and push your branch and [create a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).

You should now see a message showing your checks have failed with the merge button disabled.

![Github message showing merges blocked](/images/blog/e2e-testing/gh-merges-blocked.jpg)

Now, undo the change you made and push again and this time you should (eventually) see all green checkmarks.

![Github PR all green checkmarks](/images/blog/e2e-testing/gh-pr-all-green-checks.jpg)

You've now set up a basic E2E testing system for writing tests and preventing code regressions! <Emoji symbol="🎉" label="celebrate" /> <Emoji symbol="🎉" label="celebrate" /> <Emoji symbol="🎉" label="celebrate" />

## Final Thoughts

This wraps up the E2E Testing Series. I hope you found it useful. If you want to keep learning, I recommend watching [this video](https://www.youtube.com/watch?v=5XQOK0v_YRE), in which the founder of Cypress talks about E2E testing best practices.


To help prevent regression, we want to make sure we run our full test every time we push our code.

Ensuring Your E2E Tests Run On Every Code Push


_This post is part of the [E2E Testing Series](/blog/e2e-testing-series).  If you haven't already, I suggest going back and reading previous posts before continuing._ 

<CalloutBox>
## Just give me the code...
Here is the "tl;dr" version of the article below, with just the commands to run and the code you'll need to add.  If anything isn't clear, I suggest reading the full post.

1. Install the sample app:
   ```sh
   npx create-next-app e2e-testing --ts --use-npm
   ```
1. Create a `src` directory and move the `pages` and `styles` directories into it.
2. Install these packages:
   ```sh
   npm i -D cypress cypress-watch-and-reload start-server-and-test
   ```
2. Rename the `dev` script in your `package.json` to `dev:next`.
2. Add these scripts to your `package.json`:
      ```sh
      "cy:e2e": "cypress open --e2e --browser chrome --config baseUrl=http://localhost:3000",
   "dev": "start-test dev:next 3000 cy:e2e"
      ```
3. Run `npm run dev` to initialize Cypress.
3. Update/add these files (click on the file link to view the code):
    - [`cypress.config.ts`](https://github.com/andersr/e2e-testing-tutorial/blob/main/cypress.config.ts)
    - [`cypress/support/e2e.js`](https://github.com/andersr/e2e-testing-tutorial/blob/main/cypress/support/e2e.ts)
    - [`cypress/tsconfig.json`](https://github.com/andersr/e2e-testing-tutorial/blob/main/cypress/tsconfig.json)
    - [`tsconfig.json`](https://github.com/andersr/e2e-testing-tutorial/blob/main/tsconfig.json)
4. Run the `dev` script to enable E2E test development with hot reloading of test and source files. 
</CalloutBox>

## Introduction: Keeping E2E tests front and center
One of most common anti-patterns I see relating to E2E testing is that, while a test suite has been set up and some tests have been written, it is almost never run locally and new tests are only rarely added for new features. This obviously is not a good thing, since we effectively are wasting all the effort that went into setting up the test suite.

But why would we let such a valuable resource go to waste?

One simple answer is that it is out of sight, out of mind.  In other words, while the command to run the E2E test locally does exist, it's something you have to remember to do, and it's kind of a hassle, since you usually have to first fire up the dev environment, and then start the E2E runner in a second terminal.

One way to address this problem is to automate running of the E2E test suite whenever you start up your development environment.  As we'll see, this can have a significant positive impact on your E2E practice.

## Add a single command for starting both dev environment and test runner
To help ensure E2E tests get written, we want to make it is easy to have both our app running and the test runner in a hot-reloading state. In other words, as we write tests, we want to be able to get quick feedback on if tests are passing or not.

When we are done with the steps below, you'll be able to just run `npm run dev` to both start you dev environment and also start up your E2E test runner in dev mode, meaning that it will rerun tests automatically if changes are made either to the tests or your code. As an added bonus, it will reduce the amount of terminal windows you have open in your IDE.

## Setting up our sample app

We'll be using [NextJS](https://nextjs.org/) as the basis for our sample app. One reason for this is that our newsletter feature will require an API endpoint, and NextJS makes it super easy to add that.

To get started, go ahead and run the following to install the sample app:

```bash
npx create-next-app e2e-testing --ts --use-npm
```
You can replace `e2e-testing` with any app name you prefer. Both the `--ts` and `--use-npm` are optional.  NPM is just my personal preference, and I am a strong proponent of using TypeScript.


## Install the E2E test runner and utility packages
For this tutorial, we're going to use [Cypress](https://www.cypress.io/), which in my opinion currently is the best end-to-end testing solution. (My one gripe with Cypress is the [lack of support](https://github.com/cypress-io/cypress/issues/6423) for testing of older browsers, which are the ones often causing regressions.)

Let's install all the packages we'll need, as development dependencies: 

```bash
npm i -D cypress cypress-watch-and-reload start-server-and-test
```

### Add the dev scripts
Next, let's update the `scripts` section of our `package.json` as follows:

First, rename the `dev` script to `dev:next`.  Then, add the following scripts:

```bash
"scripts": {
    ...
    "cy:e2e": "cypress open --e2e --browser chrome --config baseUrl=http://localhost:3000",
    "dev": "start-test dev:next 3000 cy:e2e"
    ...
}
```

## Configuring Cypress 
We need to configure Cypress to use TypeScript and also to hot reload when our source files change.

### Add Typescript support
Cypress has built-in [Typescript support](https://docs.cypress.io/guides/tooling/typescript-support). To enable it, and allow for writing TS-based tests, we need to add a `tsconfig.json` to the root of the `cypress` folder.

Add the following code to `cypress/tsconfig.json` and you should now be able to create test files using Typescript.

```json
{
  "compilerOptions": {
    "target": "es5",
    "lib": ["es5", "dom"],
    "types": ["cypress"]
  },
  "include": ["**/*.ts"]
}
```

Also, update the main `tsconfig.json` in the root directory to exclude the cypress directory, so we do not compile test files as part of compiling the main source code. 

```json
{
  ...
  "exclude": ["node_modules", "cypress"]
}

```

### Get Cypress to hot reload if your source files change
By default, the Cypress runner will only re-run if you modify test files.  However, we really also want it to restart when we make changes to our code.  To achieve this, we need to make the following updates:

First, update `cypress.config.ts` as follows:

```tsx
import Cypress, { defineConfig } from "cypress";

interface Config extends Cypress.UserConfigOptions<any> {
  "cypress-watch-and-reload": {
    watch: string | string[]
  }
}

const config: Partial<Config> = {
  "cypress-watch-and-reload": {
    watch: ["src/**/*"],
  },
  e2e: {
    setupNodeEvents(on, config) {
      return require("cypress-watch-and-reload/plugins")(on, config);
    },
  },
}

export default defineConfig(config);
```
Next, add the following to the end of the file `cypress/support/e2e.ts`.

```bash
require("cypress-watch-and-reload/support");
```
Your local environment and `dev` command should now be set up.

Go ahead and run `npm run dev` and you should see the dev server first start up, then the test runner will start. The first time you run Cypress, you may be asked to click through their startup wizard.

Btw, a huge shout-out to [Gleb Bahmutov](https://glebbahmutov.com/) who is the author of both the utility packages we are using here! <Emoji symbol="👏" label="clap emoji" /> <Emoji symbol="👏" label="clap emoji" /> <Emoji symbol="👏" label="clap emoji" />

## Next Up: A Suggested Process for writing E2E tests
Since we haven't written any tests, Cypress will prompt you to add some.

![Newsletter signup form](/images/blog/e2e-testing/cypress-runner-no-specs.jpg)

In the [next part in the series](/blog/e2e-series-a-process-for-writing-tests), we'll be focusing on exactly that.


Ensure E2E tests get written by running both your dev environment and your E2E test runner with a single command. Also, learn about writing resilient tests using test ids.

How to set up a local E2E test development environment


## Introduction
This is a series of posts I've written about the process and benefits of writing and using End to End Tests (E2E). We'll walk through how to set up your system to write them, how to use the tests you've created to stay focused, and last but not least, how to use E2E tests to prevent [code regression](https://www.google.com/search?q=code+regression).

I've broken the series into the following parts:

1. **Series intro and prerequisites (current post).**
2. [Setting up your local environment for writing E2E tests](/blog/e2e-series-setting-up-your-local-environment).
2. [A suggested process for writing tests](/blog/e2e-series-a-process-for-writing-tests).
2. [Ensuring your test suite runs every time you push code](/blog/e2e-series-ensure-tests-run-on-every-push).

In this post, I'll offer a general intro to E2E testing and assumed prerequisites. I hope you find it useful.  Please feel free to post feedback in the comments.

## What are E2E tests and why should you care?
One good way to understand E2E testing is to look at what can happen without it.

### A project without E2E testing
You're working on a project that doesn't have any formal E2E testing in place.

Let's say you're adding a newsletter signup form to your web app. 

![Newsletter signup form](/images/blog/e2e-testing/newsletter-form-styled.jpg)

To keep your code [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself), you re-use existing input and form submit components to implement your form. You test the form locally and everything is working great. You also click around your web app and everything else seems to be working great as well.

However, after deploying your code, you start getting notices that users no longer can reset their passwords. <Emoji symbol="😱" label="horror emoji" />  <Emoji symbol="😱" label="horror emoji" />  <Emoji symbol="😱" label="horror emoji" /> 

But wait, you were working on newsletter signup, how could that possibly break password reset?

Actually, as it turns out, the password reset form uses some of the same components that you used when implementing newsletter signup.

![Password reset form](/images/blog/e2e-testing/password-reset-form.jpg)

When you made some tweaks to those components (maybe you decided to rename a prop) while adding the newsletter feature, you unintentionally broke the reset feature.  Worse, since that feature is out of sight, out of mind, you didn't think to test it before shipping newsletter signup.

This is an example of what we call a [code regression](https://en.wikipedia.org/wiki/Software_regression), where something that previously worked unexpectedly stops working, and it is one of the main reasons we write E2E tests.

### E2E testing to the rescue
If you'd had an E2E testing system in place, such as the one we'll be creating, and you had good feature coverage in your tests, after modifying the components being used in password reset, you'd have received a test failure notification, and then be able to quickly address the problem, preventing a reset password failure in production.

As apps get more complex and more feature-rich, and as we get better at re-using code across our apps, the risk of regression increases, so writing these types of tests and ensuring we run them regularly becomes increasingly important.

In this series, we're going to walk through the process of setting yourself up for success by enabling quickly and effectively writing these types of tests, running them locally against your development environment to get quick feedback on possible issues, and then automatically running the full suite of tests every time you push code, to prevent merging in code that will lead to regressions.

We'll also discuss how integrating E2E tests into your development process can you help you write better code overall and help ensure that you remain focused on your goals.

In my opinion, E2E tests are the most valuable test type, because they simulate actual end users and are focused on outcomes rather than implementation details. However, while valuable, these types of tests can also be costly, in that setting up your system to run them and writing the tests themselves can be complex and time-consuming. And as your test suite grows in size, they can take a long time to run.

But here's the thing: if done right, you can overcome these challenges, and get the best of both worlds. Achieving that is one of the top goals of this series.
<CalloutBox>
## A quick note about terminology
I am using the term "E2E" or End to End Testing a bit loosely here. Some might call the types of tests we are writing a functional test, or you may have a different name.  There are countless names and nuances for different types of tests, such as smoke test, integration test, regression test, and more, many of them overlapping.

It's easy to get caught up in the terminology and lose sight of the actual goals, such as writing higher quality code and preventing stuff from breaking.  I prefer just using E2E as an umbrella term for any type of test that simulates a user using the app and which treats the app itself as a black box, just as a real user would. (I.e. they don't know and don't care about the inner workings of the app.)
</CalloutBox>


## Prerequisites
- I am assuming you already have some experience with JavaScript generally and specifically with React and Node.
- I'm also assuming you already have the latest stable version of Node and npm installed on your machine.
- I'll be writing the sample code mostly in Typescript so I am assuming you are familiar with that as well.
- Finally, I am assuming you have some experience using git, as we'll be using that in the last part of the series, to create branches, pull requests, and more.

However, to be clear, many of the principles we discuss will be agnostic to whatever language or tool you're using.

## Code Repo
You can find all the source code we'll be walking through below [in this repo](https://github.com/andersr/e2e-testing-tutorial).  I recommend writing your own code rather than just copying from the repo, but you might find it to be a good reference in case you get stuck.

## Next up: Setting up your local testing environment
To get started, we'll install a sample app and set up our development environment for testing.  [Continue to the next post in the series](/blog/e2e-series-setting-up-your-local-environment).


A series of posts on how to configure and write valuable E2E tests.

Blog