Multiple browsers run tests

[JustinJohnWilliams]

Allow for multiple browsers (chrome, ff, ie) to run your test suite and report to TeamCity.

[lefthandedgoat]

I think I have a reasonable solution:

In your version of canopy locally paste this in runner.fs at the very bottom

let runFor (browsers : BrowserStartMode list) =
    let currentSuites = suites
    suites <- [new suite()]
    browsers
    |> List.iter (fun browser ->
        toString browser
        |> sprintf "Running tests with %s browser"
        |> context
        once (fun _ -> start browser)
        suites <- suites @ currentSuites)
    run ()

Then in your program.fs replace run() with this

runFor [chrome; firefox; ie]

The runFor function will take a list of browsers (start modes) and create a new context for each and open that browser. It takes the list of suites/tests you have defined and duplicates them after each browser you want.

It works for me locally. Please test it and see if it gives you the desired results. If it does I will do a nuget build!

[JustinJohnWilliams]

This works perfectly, thanks!

On Fri, Apr 3, 2015 at 11:41 AM, Chris Holt [email protected]
wrote:

I think I have a reasonable solution:

In your version of canopy locally paste this in runner.fs at the very
bottom

let runFor (browsers : BrowserStartMode list) =
let currentSuites = suites
suites <- [new suite()]
browsers
|> List.iter (fun browser ->
toString browser
|> sprintf "Running tests with %s browser"
|> context
once (fun _ -> start browser)
suites <- suites @ currentSuites)
run ()

Then in your program.fs replace run() with this

runFor [chrome; firefox; ie]

The runFor function will take a list of browsers (start modes) and create
a new context for each and open that browser. It takes the list of
suites/tests you have defined and duplicates them after each browser you
want.

It works for me locally. Please test it and see if it gives you the
desired results. If it does I will do a nuget build!

—
Reply to this email directly or view it on GitHub
#190 (comment)
.

[soerennielsen]

How about adding possibility to use browser instances instead of only browser start modes?

I use phantomjs which I need to start in a rather special way for authentication to work properly (SharePoint site using claims authentication).

Scope creeping:
Would it be possible to run these browsers in parallel?

-----Original Message-----
From: "Justin Williams" [email protected]
Sent: ‎04-‎04-‎2015 15:38
To: "lefthandedgoat/canopy" [email protected]
Subject: Re: [canopy] Multiple browsers run tests (#190)

This works perfectly, thanks!

On Fri, Apr 3, 2015 at 11:41 AM, Chris Holt [email protected]
wrote:

I think I have a reasonable solution:

In your version of canopy locally paste this in runner.fs at the very
bottom

let runFor (browsers : BrowserStartMode list) =
let currentSuites = suites
suites <- [new suite()]
browsers
|> List.iter (fun browser ->
toString browser
|> sprintf "Running tests with %s browser"
|> context
once (fun _ -> start browser)
suites <- suites @ currentSuites)
run ()

Then in your program.fs replace run() with this

runFor [chrome; firefox; ie]

The runFor function will take a list of browsers (start modes) and create
a new context for each and open that browser. It takes the list of
suites/tests you have defined and duplicates them after each browser you
want.

It works for me locally. Please test it and see if it gives you the
desired results. If it does I will do a nuget build!

—
Reply to this email directly or view it on GitHub
#190 (comment)
.

—
Reply to this email directly or view it on GitHub.

[lefthandedgoat]

I will work on getting instance to work also. It shouldn't be hard.

For parallel there is a problem. 'browser' is a static instance of the current browser that is being used. Since there can be only one all your parallel tests would be fighting over the browser and it would not play nice.

I havent tried it but this may be a solution

https://msdn.microsoft.com/en-us/library/system.threadstaticattribute%28v=vs.110%29.aspx

If I start each parallel run in its own thread it seems like it would have an instance of the browser per thread and not cause conflicts. At the moment I haven't tried it. Do you have any knowledge of how ThreadStatic works?

[soerennielsen]

I haven't tried the ThreadStatic attribute but it looks pretty
straightforward.

I think that there is a need for some refactoring to make it work. The
browser attribute will need to be instantiated within each thread instead
of on the main thread - we also need to ensure that the reporter is
threadsafe and possibly some of the suite variables as well. Shouldn't be
too hard.

I will give it a go, however it might take me a little while ...

On 4 April 2015 at 20:41, Chris Holt [email protected] wrote:

I will work on getting instance to work also. It shouldn't be hard.

For parallel there is a problem. 'browser' is a static instance of the
current browser that is being used. Since there can be only one all your
parallel tests would be fighting over the browser and it would not play
nice.

I havent tried it but this may be a solution

https://msdn.microsoft.com/en-us/library/system.threadstaticattribute%28v=vs.110%29.aspx

If I start each parallel run in its own thread it seems like it would have
an instance of the browser per thread and not cause conflicts. At the
moment I haven't tried it. Do you have any knowledge of how ThreadStatic
works?

—
Reply to this email directly or view it on GitHub
#190 (comment)
.

[lefthandedgoat]

Ok cool, I have some ideas in my head too try to make it work (other than thread static). I will try to do some proof of concept works around that.

Tonight I will change the runFor method to also accept already running browsers instances so you can try that out too.

[lefthandedgoat]

@soerennielsen try this. For me it works for already started browsers or start modes.

let runFor browsers =
    let currentSuites = suites
    suites <- [new suite()]
    match box browsers with
        | :? (types.BrowserStartMode list) as browsers -> 
            browsers
            |> List.iter (fun browser ->
                toString browser
                |> sprintf "Running tests with %s browser"
                |> context
                once (fun _ -> start browser)
                suites <- suites @ currentSuites)
        | :? (IWebDriver list) as browsers ->
            browsers
            |> List.iter (fun browser ->
                browser.ToString()
                |> sprintf "Running tests with %s browser"
                |> context
                once (fun _ -> switchTo browser)
                suites <- suites @ currentSuites)
        | _ -> raise <| Exception "I dont know what you have given me"
    run ()

[lefthandedgoat]

This is available now in the latest nuget.

https://www.nuget.org/packages/canopy/0.9.23

[manishdube]

@lefthandedgoat where should the drivers for the multiple browsers live? We have it setup so that there is a "driver" folder in the solution where the chrome driver is at. Is this where we would put the firefox and IE drivers too ?

[lefthandedgoat]

@manishdube You can, I use nuget packages to manage drivers

https://www.nuget.org/packages/Selenium.WebDriver.ChromeDriver/85.0.4183.3800-beta
https://www.nuget.org/packages/Selenium.WebDriver.GeckoDriver/

[manishdube]

@lefthandedgoat thanks again. I installed the packages but when in Program.fs i try to replace

run()

with
runFor [chrome; firefox]

i get an error FS0001: This expression was expected to have type� 'Browsers' �but here has type� ''a list'

[lefthandedgoat]

You have to use one of these types:

canopy/src/canopy/types.fs

Line 117 in cfe3fa1

| BrowserStartModes of BrowserStartMode list

[lefthandedgoat]

Try

runFor (BrowserStartModes [chrome; firefox; ie])

[manishdube]

when i comment

// run()

and use

runFor (BrowserStartModes [chrome; firefox; ie])

i still see compile errors :| Sorry Im being embarrassing i know but not sure how to get past this? ... @lefthandedgoat

[olivercoad]

@manishdube what version of canopy and dotnet are you using? What are the compile errors?

[manishdube]

.NET Framework 4.7 and Canopy 2.1.5 @olivercoad
Opens up the console prompt, the chrome HTML reporter and an empty chrome instance and goes to quit()
`
namespace Main

module Program =

open canopy.runner.classic
open canopy.configuration
open canopy.classic
open canopy.reporters
open canopy.types

open System
open TestRunner
open ArguParser

[<EntryPoint>]
let main argv =
    //for i in 1u .. 1u do

        //Parse all the args into the types that we use in the rest of the code
        let args = Args.parse argv

        chromeDir <- (AppDomain.CurrentDomain.BaseDirectory + "\drivers")
        elementTimeout <- 30.0

        let chromeOptions = OpenQA.Selenium.Chrome.ChromeOptions()
        chromeOptions.AddAdditionalCapability("useAutomationExtension", false);

        reporter <- new LiveHtmlReporter((ChromeWithOptions chromeOptions), chromeDir) :> IReporter
        reporter.setEnvironment "GFS TEST"

        let liveHtmlReporter = reporter :?> LiveHtmlReporter
        let htmlReporterPath = sprintf "logs\TestSummary-%s" <| DateTime.Now.ToString "yyyy-MM-dd-HHmm"
        liveHtmlReporter.reportPath <- Some(htmlReporterPath)

        //Start the browser supplied in args
        start args.Browser
        browser.Manage().Window.Maximize()

        //Register the tests that you want to run (under development, a specific page, all tests, etc)
        RegisterTests.register args.Tag args.TestType

        //Run tests

        //run()

        runFor (BrowserStartModes [chrome; firefox])
    
        //Quit all browsers
        quit()
    
        //return code
        canopy.runner.classic.failedCount

// 0

`

[olivercoad]

@manishdube not sure what TestRunner and ArguParser are so can't try your code. Are they other files in your project?

If you look at the implementation of runFor, it replaces the list of suites with a copy for each browser but doesn't actually run them. @lefthandedgoat at some point it must've lost the call to run () at the end.

Try doing

runFor (BrowserStartModes [chrome; firefox])
run ()

[manishdube]

Thanks @olivercoad your suggestion worked.

runFor (BrowserStartModes [chrome; firefox]) run ()

[manishdube]

anyway this could be extended to the browser Edge ?
If yes, what NuGet package do you use for that ? Thanks.