Authenticated workflow testing must prove that the right user can sign in and finish the protected task after a deployment. AlwaysQA tests password and six-digit email-code sign-in through a fresh browser session, giving the team visibility into the complete workflow without giving the decrypted password, QA Inbox, Email Code, or raw authentication message to a Coding Agent or browser agent.
Most teams make the mistake of treating login as a checkpoint. The expensive failures happen across the whole chain: form, credential submission, identity-provider redirect, session, permissions, and the product action the customer came to complete.
Reality check: A green login page proves that the page loaded. It doesn’t prove that an authenticated customer can use the product.
The use case: verify the job behind the login
Consider a SaaS team that has just shipped a small permissions change. Public pages respond normally. The sign-in form loads. Infrastructure monitoring is green.
But standard users now return to the login page after entering a valid password. Administrators are unaffected, so the problem survives an internal spot check. Customers discover it first.
A useful Test Case doesn’t stop at “login succeeded.” It verifies a visible product result:
Starting URL
https://app.example.com/loginInstructions Sign in as the configured standard user. Create a test project named with today’s date. Confirm that it appears in the project list, then delete it.
Success Condition The project appears after creation and no longer appears after deletion.
That Run covers the sign-in form, credential acceptance, redirect, authenticated session, role permissions, record creation, persistence, and cleanup. It answers a business question, not a transport-level one.
For broader coverage across signup, onboarding, permissions, billing, and other release risks, use the web app regression testing checklist. This page focuses specifically on authentication and the workflows gated by it.
Why URL checks and login-only tests miss real failures
Authentication is a dependency chain. A failure at any layer can block the customer while shallow monitoring still reports success.
| Layer | What a shallow check sees | What the customer experiences |
|---|---|---|
| Login page | The URL returned 200 OK | The submit action does nothing |
| Password submission | The form accepted input | A valid credential is rejected |
| Email delivery | The email service is available | The current six-digit code never arrives |
| Code verification | The form accepts six characters | A fresh code is rejected or expires |
| Hosted authentication | The identity provider responded | The redirect returns to the wrong origin |
| Session | Authentication completed | The next page asks the customer to sign in again |
| Authorization | The dashboard opened | The account has the wrong role or permissions |
| Product workflow | Login passed | The customer still can’t complete the task |
Customers don’t buy access to a login screen. They buy what the authenticated product lets them do.
Use dedicated QA Accounts, not real customer credentials
AlwaysQA uses a QA Account: a named, Project-owned authentication resource for a dedicated identity in your application. Multiple Test Cases in the same Project can reuse it.
Create separate QA Accounts for the roles you need to verify, such as standard user, administrator, or billing manager. Give each one only the permissions and test data required for its workflows. That reduces operational risk and makes a failure easier to interpret. It also follows the least-privilege principle described in NIST SP 800-53 Revision 5.
Each QA Account has one explicit, immutable Authentication Method:
| Authentication Method | What you configure | When to use it |
|---|---|---|
| Password | A dedicated identifier and password | The application signs in with username or email plus password |
| Email Code | A dedicated identity using an AlwaysQA-controlled QA Inbox | The application sends an exact six-digit code by email |
AlwaysQA doesn’t infer the method from the page. If the product supports both methods, create two QA Accounts and assign each Test Case to the appropriate one.
The QA Account feature overview explains how this setup supports role-specific and permission-specific workflows.
The trust boundary: the agent navigates, trusted code authenticates
The browser agent needs to understand the interface. It doesn’t need the authentication secret.
During a Run, the agent navigates to the matching sign-in form and requests one QA Account Sign-in action. Trusted code completes the configured Authentication Method and returns only a sanitized result. After successful sign-in, the browser agent continues with the Test Case instructions.
The boundary is easier to see as a handoff:
Browser agent Trusted sign-in action Application
| | |
| navigate to login | |
|-------------------------->| |
| | submit protected material |
| |-------------------------->|
| | sanitized result |
|<---------------------------| |
| continue product workflow |
|------------------------------------------------------->|What stays outside agent context:
| Authentication material | Handled by | Given to Coding Agent or browser agent? | Retained as Run Evidence? |
|---|---|---|---|
| Decrypted password | Trusted sign-in action | No | No |
| QA Inbox address | Authenticated user and trusted code | No | No |
| Raw authentication message | Trusted code | No | No |
| Six-digit Email Code | Trusted code | No | No |
| Sanitized sign-in result | Trusted code and Run | Yes | Only sanitized observations |
It’s the product’s core security distinction: the agent can evaluate the authenticated experience without receiving the material used to enter it.
It’s still a dedicated identity, and it still needs operational discipline. Use synthetic data, minimum permissions, a cleanup strategy, and an owner who can rotate or remove the identity when the application changes.
How password authentication works
For a Password QA Account, configure the identifier, password, Login Page URL, and approved Login Origin directly through AlwaysQA.
Every Run starts with fresh browser state. The browser agent reaches the visible sign-in interface, then trusted code submits the configured authentication material. The agent receives a sanitized result and continues only after the sign-in action completes.
The password is write-only from the perspective of product read interfaces. It isn’t returned to your Coding Agent, placed in its prompt, or exposed to the browser agent.
That’s different from instructing an AI agent to retrieve a secret from an environment variable and type it into a page. The agent controls the workflow; AlwaysQA controls the credential boundary.
How six-digit email-code authentication works
Passwordless testing is harder because “use the latest email” isn’t a reliable rule. An inbox may contain a code from an earlier attempt, delayed mail, or multiple messages that look valid.
Each Email Code QA Account receives one stable, opaque QA Inbox. Its address contains no User, Project, or QA Account name. The authenticated user copies that address when provisioning the dedicated identity in the application; the Coding Agent and browser agent never receive it.
For each unattended Run, AlwaysQA:
- opens a fresh browser session;
- navigates to the matching sign-in form;
- opens one short-lived authentication attempt;
- requests one authentication message;
- accepts only a fresh message delivered after that request;
- extracts one exact six-digit Email Code through trusted code;
- submits that code on the approved Login Origin; and
- continues with the authenticated Test Case.
The Run doesn’t reuse an old code, request an automatic resend, or try several candidates after rejection.
Runs using the same Email Code QA Account execute one at a time. Serialization prevents overlapping messages and product activity from being attributed to the wrong Run. Other Test Cases remain independent.
Each Email Code QA Account receives an AlwaysQA-controlled QA Inbox. Your company mailbox isn’t part of the workflow. When trusted processing ends, the provider copy of the raw authentication message is deleted. The payload isn’t stored in the AlwaysQA database, AI context, logs, or Run Evidence; only sanitized diagnostics such as delivery timing and failure classification remain.
Deep dive: approve the exact destination before submitting a secret
Keeping credentials out of an AI prompt solves only half the problem. Trusted code must also know where it is allowed to submit them.
An application may start sign-in here:
https://app.example.com/loginand collect the password or Email Code here:
https://auth.example-provider.comThey’re different origins. The Login Page URL tells AlwaysQA where to begin navigating; it isn’t permission to submit authentication material.
During setup, AlwaysQA follows the visible flow and detects the exact public HTTPS Login Origin that would receive the identifier or credential. The authenticated user must review and approve that origin before the QA Account becomes Ready.
If a later Run reaches a different origin, AlwaysQA stops before submitting authentication material. The QA Account moves to Needs Attention, dependent Schedules pause, and the team must approve the new origin before those Runs can continue.
This protects against an unexpected redirect receiving stored authentication material. It doesn’t replace an application-security review. Teams should separately assess authenticator lifecycle, session management, recovery, and abuse controls against guidance such as NIST SP 800-63B.
Where AlwaysQA fits alongside Playwright or Cypress
Keep your existing lower-level automated coverage and scripted browser tests. They’re the right choice for fast deterministic assertions in a development pipeline.
AlwaysQA covers a different operating need: post-deploy QA that a Coding Agent can create, run, inspect, and use during fix-and-rerun work without receiving the QA Account’s authentication material. The AlwaysQA workflow overview shows how Test Cases, Runs, Outcomes, and Evidence connect.
| Keep a scripted browser test when… | Use AlwaysQA when… |
|---|---|
| The team needs deterministic assertions on implementation details | The team needs to verify an observable customer outcome |
| The check belongs on every commit | The check should run after a change, on demand, Daily, or Weekly |
| The team owns and maintains the test code and selectors | The workflow is defined through instructions and a Success Condition |
| Existing CI infrastructure already protects its secrets | The Coding Agent should operate QA without retrieving the credential |
| A machine-readable failure is enough | The team needs observations, a timeline, replay context, and an Issue Report |
The strongest setup isn’t either/or. Scripted tests catch known implementation failures quickly. AlwaysQA verifies that the complete critical flow still works after the change reaches the application.
Separate a broken product from an inconclusive Run
Every finished Run receives one of three Outcomes:
- Passed: the Success Condition was observed.
- Failed: valid Evidence showed that the expected application behavior was broken.
- Needs Attention: AlwaysQA couldn’t reach a trustworthy conclusion.
The third Outcome matters in authenticated testing. A missing email, delayed delivery, expired Email Code, bot protection, or unclear result shouldn’t be reported as proof that the product failed.
Some problems affect only the current Run. If CAPTCHA appears, for example, the Run ends as Needs Attention without asking AI or trusted code to bypass it, while the QA Account remains Ready.
Other problems require account-level action. An explicitly rejected password or a changed Login Origin can move the QA Account to Needs Attention, make it unavailable for new Runs, and pause dependent Schedules until the user resolves it.
AlwaysQA preserves structured Evidence: a summary, browser observations, a compact action timeline, the final Outcome, guidance when action is required, and a temporary browser replay when available. A Failed Run can also produce an editable Markdown Issue Report without including QA Account secrets, Email Codes, or raw authentication messages.
Use deployment QA history to connect that Evidence to the affected check version, fix the problem, and rerun the same Test Case.
Current authentication coverage
The safe product promise is a precise one.
| Supported now | Not supported now |
|---|---|
| Username or email plus password | Email sign-in links |
| Exact six-digit codes delivered by email | SMS codes |
| Hosted authentication pages with an approved Login Origin | TOTP authenticator codes |
| Fresh sign-in during every Run | Reusing stored authenticated browser state |
| Observable post-login browser workflows | SSO or OAuth popups |
| On-demand, Daily, or Weekly Runs | Passkeys |
| Human approval during a Run | |
| CAPTCHA bypass |
These boundaries are useful qualification criteria. If a workflow needs a magic link, SMS, passkey, OAuth popup, or human approval, use a different testing approach for that path today.
Put one authenticated workflow under post-deploy QA
Start with the protected task whose unnoticed failure would cost the team most. Login by itself is usually too narrow.
Before scheduling the Test Case:
- create a dedicated identity with minimum permissions;
- select Password or Email Code explicitly;
- approve the exact HTTPS Login Origin;
- use synthetic data that the Run can safely change;
- choose a workflow with a visible, business-relevant result;
- define cleanup for records the Run creates;
- make one person responsible for Failed and Needs Attention Outcomes; and
- run the Test Case once on demand to review its Evidence.
Then add a Daily or Weekly Schedule when the workflow and test data are stable. For deployment-specific verification, run the Test Case again after the relevant change ships.
Create your AlwaysQA account and put the authenticated workflow your customers depend on under post-deploy QA.
Frequently asked questions
Does AlwaysQA expose passwords or Email Codes to AI agents?
No. Trusted code handles the configured password, raw authentication message, and six-digit Email Code. The Coding Agent and browser agent receive only the non-secret workflow context and sanitized sign-in result they need.
Does AlwaysQA need access to an existing company mailbox?
No. Each Email Code QA Account uses an AlwaysQA-controlled QA Inbox instead of connecting to an existing company mailbox.
Can one QA Account test password and email-code authentication?
No. Each QA Account has one immutable Authentication Method. Create separate Password and Email Code QA Accounts and assign each Test Case explicitly.
What happens when the hosted authentication origin changes?
AlwaysQA stops before submitting authentication material, moves the QA Account to Needs Attention, and pauses dependent Schedules. The user must review and approve the new Login Origin before Runs can continue.