Skip to content

Nala (E2E UI Test Automation Framework)

Jump to bottom
Santoshkumar Nateekar edited this page Nov 25, 2024 · 3 revisions

NALA Introduction

Automated E2E UI testing of Milo-based projects.

  • End-to-End UI Automation Framework for Milo and Milo-based Projects
  • Built around the Microsoft Playwright tool
  • Easy to create UI automation test scripts by anyone
  • Provides flexibility to run and debug tests in the developer environments (local, libs, and feature branches).
  • Offers the capability to run tests on various platforms and browser combinations (Windows, Linux, MacOS, & Chrome, Firefox, Web Kit, etc.).
  • Supports running UI tests on PR branches via GitHub Actions, Circle Ci, and Jenkins for PR approvals

Nala Directory Structure

NALA folder contains all UI tests

  • Block folder: Contains tests for all Milo blocks.

  • Feature folder: Contains tests for all Milo features.

  • Libs folder: Contains web and other test/validation libraries

  • Util folder: Contain utility methods like a reporter, slack integrations, etc.

    image

Nala Test Structure

NALA test contains 3 main files

  • .page.js:

    • Contains page object class  block and features
    • Contains locators for all web elements

  • .spec.js:

    • Contains test cases or spec
    • Includes test pages and test data

  • .test.js:

    • Validates the test cases/specs
    • Runs on different platforms or browsers ( UI, Debug and Headless mode)

    image

Nala Tests

  • .page.js (page object):

    image

  • .spec.js (test case/feature spec):

    image

  • .test.js (test script):

    image

Nala test creation

NALA Test creation is Quick and Easy:

  • Test Scripts: For the first time, create the following files, and then update or add specs and tests/test steps

    • .page.js for page selectors/ locator
    • .spec.js for adding test cases/specs
    • .test.js for validating test case/specs

  • Code Snippets: Use below code snippets commands to create required page objects, specs, and tests

    • create nala page
    • create nala spec
    • add nala spec
    • create nala test
    • add nala test

  • Test Pages:

    • Test pages needs to created under draft>>nala folder in share point.

Nala test run

NALA tests can be run in the following modes on multiple browsers (Chrome, Firefox, Web Kit, etc.), platforms (Linux, Windows, macOS, etc.), and application environments ( local, libs, stage, main, feature-branch, etc.).

  • Headless mode

  • UI Mode

  • Debug mode

  • Running Tests

    • Run/debug tests on localhost (https://localhost:3000/)
    • Run/debug tests on libs (https://localhost:6456)
    • Run/debug tests on feature branch ( https://feature-branch--milo--adobecom/)
    • Run/debug test with ?milolibs=<>
    • Run tests on the PR branch ( GitHub Actions)
    • Run tests on stage, main branches ( GitHub Actions)
    • Run test on milolibs ( GitHub Actions)
    • Run test on Circle Ci ( inside adobe n/w)

Nala Run command

  • npm run nala <env> [options]
    • Env: [local | libs | stage | etc. ] default: local
    • Options:
      • browser=<chrome|firefox|webkit> Browser to use (default: chrome)
      • device=<desktop|mobile> Device (default: desktop)
      • test=<.test.js> Specific test file to run (runs all tests in the file)
      • -g, --g=<@tag> Tag to filter tests by annotations ex: @test1 @accordion @marquee
      • mode=<headless|ui|debug|headed> _Mode (default: headless) _
      • config= Configuration file (default: Playwright default)
      • project= _Project configuration (default: milo-live-chromium)
      • milolibs=<local|prod|feature|any|> _Milolibs?=

.

Nala Accessibility Testing

Accessibility Testing

  • Evaluate web or mobile apps for accessibility by users with diverse abilities and disabilities

  • The goal is to provide an inclusive, equitable experience for all users, regardless of physical, sensory, cognitive, or technological & compliance with global accessibility guidelines and legal requirements across the regions.

Key aspects of Accessibility testing

  • Screen Reader Compatibility
  • Keyboard Navigation
  • Color Contrast
  • Text Alternatives
  • Scalable Text and Layout.
  • Consistent and Predictable Design.
  • Testing Against Accessibility Standards, Guidelines and Legal Compliances (Examples : Americans with Disabilities Act (ADA), Section 508 of the Rehabilitation Act, European Accessibility Act (EEA), WCAG 2.0/2.1 A, 2.0/2.1 AA, 2.0/2.1 AAA (Web Content Accessibility Guidelines) etc.)

WCAG (Web Content Accessibility Guidelines)

  • WCAG is an international standard for Web Accessibility
  • Developed by W3C (World Wide Web) process, it provides a set of recommendation and guidelines to make web content more accessible •WCAG has three levels of compliance
    1. WCAG 2.0/2.1 A: Basic web accessibility features
    1. WCAG 2.0/2.1 AA: More inclusive features that cover a wide range of disabilities (most common level for legal requirement)
    1. WCAG 2.0/2.1 AAA: The highest level with the most comprehensive accessibility features

Accessibility Compliance Standards Across Regions

image

NALA Accessibility Scope

Uses @axe-core/playwright library for accessibility testing

Accessibility Testing Scope

    1. Milo Sections / Blocks Level: Scans section or blocks against WCAG guidelines for accessibility validations
    1. Entire Web Page Level: Scanning the entire web page against WCAG guidelines for accessibility validations

Accessibility test method/function:

image

Accessibility Test Step: All Nala automation scripts for the Milo application include an accessibility test step that scans for accessibility violations at sections/block

image

GitHub Action Run

image

Nala Accessibility Test Commands

  1. Run an accessibility test on any URL / page,
  • npm run a11y url
  • ex: npm run a11y https://developer.adobe.com
  1. Run an accessibility test on any section of a page ( ex, particular section, milo blocks, or any web element)
  • npm run a11y url -- -s='body'
  • examples:
    • npm run a11y https://developer.adobe.com -- -s='body'
    • npm run a11y https://developer.adobe.com -- -s='.section'
    • npm run a11y https://developer.adobe.com -- -s='.accordion'
  1. Run accessibility tests against any WCAG tags, such as wcag21a, wcag21aa, and others
  • npm run a11y url -- -t=wcag2a
  • ex:
    • npm run a11y https://developer.adobe.com -- -t=wcag2a
    • npm run a11y https://developer.adobe.com -- -t='wcag2a','wcag2aa'
  1. Run accessibility tests on multiple URLs by adding them to a .txt file
  • npm run a11y url -- -f=filepath.txt
  • ex:
    • npm run a11y https://developer.adobe.com -- -f=path.txt
  1. Run accessibility tests on any of the milo envs
  • npm run a11y [env] [page path]
  • ex:
    • npm run a11y stage /drafts/nala/accordion/accordion

Nala Accessibility HTML Report

  • image
  • image

Nala Accessibility Terminal Debug / Report

image
Clone this wiki locally