Saturday, March 22, 2014

Full automation of Protractor E2E tests using Grunt - Part 2

This is a continuation of the article series on automating protractor tests using grunt. You can see the first part here

In this part we will be doing the following tasks.

  • Adding the setup tasks to grunt to make the whole process easier to start. 
  • Adding an HTML reporter

Adding the setup command tasks


When you create a repo, you may want to clone this repo on several machines. Or, when someone else wants to use your repo to use in his/her project. When they do that, they also have to go through all the basic setup that you have done like installing selenium, chromedriver etc. We don't want that. 

We are doing all our setup on the command console. Then, why not automate it? Is there any grunt plugin for that? Yes. There is. We can use grunt-shell-spawn for running all the shell commands.

Lets add grunt-shell-spawn module to our project.



Now, we will add the shell-spawn task to Gruntfile.


   You can see that I have added two options. 'protractor_install' and 'npm_install'. It is better to add an npm install before we are installing chrome driver and selenium just to make sure that, everything is ready beforehand.

  Now we will add the install task. I have added both 'npm_install' and 'protractor_install' to the install task.




Now, if we do a grunt install on the command console, it will show as everything uptodate, since we have already installed selenium and chromedriver in the current project folder.


PS: You can see some warning. Please ignore that for now. This is because we haven't added the README.md file and repo to the project, which we will add at the end.

To demonstrate the usefulness of what we did now, lets make a copy of our project folder. Make sure that, you are not copying the node_modules folder. I copied the files to 'exampleCopy' folder. Go to this copy folder in the command console.

Now, if we do the grunt install, everything should work, right? Let's see.



It is saying that a local copy of grunt is required. So, we have to do an npm install before we call any grunt tasks. Let's do that.

1. npm install

2. grunt install


Now you are ready to do the protractor tests. So, any other person using this repo only need to run two commands "npm install" and "grunt install" and they are ready to go.

Adding the HTML Reporter

When we run the tests, we can see that tests are passing or failing in the command console. What if we have hundreds of tests and also want to keep a record of the test results  whenever we run the tests? We need test reporters for that.

Protractor gives the options to add onPrepare function property on its configuration file. This onPrepare function will be called before each tests. Jasmine test framework(which we are using inside protractor) provides the option to add test reporter. We will get the metadata about each test by using these two configurations.

There is a node module available for generating screenshots for each tests. protractor-screenshot-reporter. However this doesn't process the meta data and gives us the result in a readable format.

I couldn't find a node module doing this for protractor. So, I created a module for generating this HTML report, on top of protractor-screenshot reporter, protractor-html-screenshot-reporter. What it does is generate an HTML report with all the details of the test like status, browser used, message, link to screenshot etc. (If you are finding some other modules which are doing the same, please add in the comments. )

Lets install protractor-html-screenshot-reporter module in our project.


We need to add this reporter in the Protractor configuration file.

1. require the protractor-html-screenshot-reporter module and assign to a variable.


2. add this reporter on the onPrepare function.


onPrepare: function() {
      // Add a reporter and store screenshots to `screnshots`:
      jasmine.getEnv().addReporter(new HtmlReporter({
         baseDirectory: 'screenshots'
      }));
   },

3. Run the test using grunt command. It will run the default task in the Gruntfile.



   We can see that the tests are passing. Lets see if the reports are generated.

Screenshots folder is created.


Inside the screenshots folder, one json and png with a guid filename is available. This is the metadata and screenshot respectively for the single test. There is also a combined.json which combines all the metadata into a single file and a reporter.html file which shows all the test reports.


Opening the reporter.html looks like below.



Clicking on View link opens the screenshot image also.




So, now you have an html reporter ready. But, there is one issue though. Whenever you run the tests, the reporter files are getting overridden. If I want to keep a track of tests, this is not going to help.

protractor-html-screenshot-reporter provides the option of pathBuilder function property to give your own dynamic paths. We will add this in Protractor configuration file.



  onPrepare: function() {
      // Add a reporter and store screenshots to `screnshots`:
      jasmine.getEnv().addReporter(new HtmlReporter({
         baseDirectory: 'screenshots',
         pathBuilder: function pathBuilder(spec, descriptions, results, capabilities) {
          
            var monthMap = {
              "1": "Jan",
              "2": "Feb",
              "3": "Mar",
              "4": "Apr",
              "5": "May",
              "6": "Jun",
              "7": "Jul",
              "8": "Aug",
              "9": "Sep",
              "10": "Oct",
              "11": "Nov",
              "12": "Dec"
            };

            var currentDate = new Date(),
                currentHoursIn24Hour = currentDate.getHours(),
                currentTimeInHours = currentHoursIn24Hour>12? currentHoursIn24Hour-12: currentHoursIn24Hour,
                totalDateString = currentDate.getDate()+'-'+ monthMap[currentDate.getMonth()]+ '-'+(currentDate.getYear()+1900) + 
                                      '-'+ currentTimeInHours+'h-' + currentDate.getMinutes()+'m';

            return path.join(totalDateString,capabilities.caps_.browserName, descriptions.join('-'));
         }
      }));
   },


I am just using a basic function to give the folder name as the current date and time and browser.

Note that I am using the path module for creating the file path (path is a native node module for handling and tranforming file paths. ). Make sure that, you have required the path and assigned to a variable.





Lets run the test again using grunt command in command console. As expected tests are running successfully.

If we check the project folder we can see that, the report is now generated in the path 'screenshots/22-Feb-2014-5h-58m/chrome'. Great. Now, on seeing the folder names itself, you know when the test was run and on which browser.


Finally, lets make this project ready to add as a git repo. 

1. It is not a good a good practice to push node_modules directory to the repo. In order to avoid node_modules to be pushed to git, we need to add it to .gitignore file. 

Create a new file in the root folder with name as .gitignore. 



We don't want screenshots and node_modules folder to be added in git. So, I have added both names in .gitignore. 

2. We will also add the repository details to package.json. 



3. Also, create a README.md file in the root folder and add some description of your project. 

Finally, we are ready to push to the repo. 

More Requirements

 Now that we are in a good state, we can check the further requirements.

  1. Generating tests based on style guide
  2. Doing CSS Regression testing

We will cover that in the coming articles. Stay tuned.

Please share your comments.

All the changes are added on github repo - protractor-e2e-bootstrap


22 comments:

  1. Reporting of month is off by One month(Dec(12) is reported as Nov). Reporting of OS is also wrong, Windows 8.1 is reported as XP

    ReplyDelete
    Replies
    1. Reporter gets the metadata using the browser.getCapabilities() method provided by protractor which internally calls webdriver.getCapabilities method. You can get more details by raising an issue in protractor repo.

      Delete
    2. Hi Jose,

      Thanks for the information. However is there any way to override the OS name? Or could you please direct me to right place where I can raise this issue?

      Thanks,
      Swarup

      Delete
  2. Replies
    1. Have you raised an issue on the github repo?

      Delete
  3. There is also an issue with the jasmine call in the function definition:
    ReferenceError: jasmine is not defined

    ReplyDelete
    Replies
    1. Have you raised an issue on the github repo?

      Delete
  4. We can avoid the reports overridden issue by parameterize the screen path along with the timestamp as below:

    var HtmlReporter = require('protractor-html-screenshot-reporter');
    var path = require('path');
    var currentdate = new Date();

    var screenpath = 'Reports/'+ currentdate.getDate() + "-" + (currentdate.getMonth()+1) + "-" + currentdate.getFullYear() + " @ "
    + currentdate.getHours() + currentdate.getMinutes() + currentdate.getSeconds();
    exports.config = {
    seleniumAddress: 'http://localhost:4444/wd/hub',
    specs: ['e2e/*.spec.js'],
    baseUrl: 'https://..../',
    jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000
    },
    onPrepare: function() {
    // Add a screenshot reporter:
    jasmine.getEnv().addReporter(new HtmlReporter({
    /*baseDirectory: function crDir(){
    var currentdate = new Date();
    var datetime = currentdate.getDate() + "-"
    + (currentdate.getMonth()+1) + "-"
    + currentdate.getFullYear() + " @ "
    + currentdate.getHours() + currentdate.getMinutes() +
    currentdate.getSeconds();
    var screenpath = 'screenshots'+ "_" + datetime ;

    return screenpath;
    },*/
    baseDirectory: screenpath,
    takeScreenShotsOnlyForFailedSpecs: true,
    pathBuilder: function pathBuilder(spec, descriptions, results, capabilities) {

    var monthMap = {
    "1": "Jan",
    "2": "Feb",
    "3": "Mar",
    "4": "Apr",
    "5": "May",
    "6": "Jun",
    "7": "Jul",
    "8": "Aug",
    "9": "Sep",
    "10": "Oct",
    "11": "Nov",
    "12": "Dec"
    };

    var currentDate = new Date(),
    currentHoursIn24Hour = currentDate.getHours(),
    currentTimeInHours = currentHoursIn24Hour>12? currentHoursIn24Hour-12: currentHoursIn24Hour,
    totalDateString = currentDate.getDate()+'-'+ monthMap[currentDate.getMonth()+1]+ '-'+(currentDate.getYear()+1900) +
    '-'+ currentTimeInHours+'h-' + currentDate.getMinutes()+'m';

    return path.join(totalDateString,capabilities.caps_.browserName, descriptions.join('-'));
    }
    }));
    },
    };

    ReplyDelete
  5. Nice information about the load testing!!! I prefer Loadrunner automation testing tool to validate the performance of software application/system under actual load. Loadrunner training institute in Chennai | Loadrunner course in Chennai

    ReplyDelete
  6. Jinto,

    I used the same configurations which u mentioned above but I haven't seen any report or screenshort generated for my scenario, also the folder i manually created for storing the reports get deleted when i ran my script. Please let me know ur thoughts on this!!

    ReplyDelete
  7. WebMethods Training in Chennai
    This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic..

    ReplyDelete
  8. There is a better way of doing protractor screenshots, have a look at https://github.com/azachar/protractor-screenshoter-plugin (disclaimer: I am the author)

    ReplyDelete
  9. There is a better way of doing protractor screenshots, have a look at https://github.com/azachar/protractor-screenshoter-plugin (disclaimer: I am the author)

    ReplyDelete
  10. Wow, the information on this site has helped ne to finish writing my thesis paper and I am grateful that I landed on it while I was looking for Urgent Help to Finish a Project. I will bookmark this site so that I can visit it occasionally to read more programming information. I wonder where I could have found such information if it was not posted on this site.

    ReplyDelete
  11. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  12. It is extraordinarily a magnificent work and the way in whatever u r sharing the experience information is excellent.Thanks for treating me to understand basic concepts.If want become learn for Java Training with OOPS knowledge to reach us Besant Technologies.To click the training details, Java Training in Chennai | Java Training Institute in Chennai

    ReplyDelete
  13. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  14. Nice Article, But I want to know that, there is any way to run that report.html file on browser automatically after creation of this file.

    ReplyDelete
  15. Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  16. really good piece of information, I had come to know about your site from my friend shubodh, kolkatta,i have read atleast nine posts of yours by now, and let me tell you, your site gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanks a lot once again.
    Hadoop Training in Chennai
    Big Data Training in Chennai
    Python Training in Chennai
    Python Training Centers in Chennai
    Data Science Training in Chennai
    Data Science Course in Chennai
    Data Analytics Training in Chennai
    Best AngularJS Training in Chennai
    AngularJS Training in Chennai
    QlikView Training in Chennai
    Informatica Training in Chennai

    ReplyDelete