Saturday, March 22, 2014

Full automation of Protractor E2E tests using Grunt - Part 2

This is a continuation of the article series on automating protractor tests using grunt. You can see the first part here

In this part we will be doing the following tasks.

  • Adding the setup tasks to grunt to make the whole process easier to start. 
  • Adding an HTML reporter

Adding the setup command tasks


When you create a repo, you may want to clone this repo on several machines. Or, when someone else wants to use your repo to use in his/her project. When they do that, they also have to go through all the basic setup that you have done like installing selenium, chromedriver etc. We don't want that. 

We are doing all our setup on the command console. Then, why not automate it? Is there any grunt plugin for that? Yes. There is. We can use grunt-shell-spawn for running all the shell commands.

Lets add grunt-shell-spawn module to our project.



Now, we will add the shell-spawn task to Gruntfile.


   You can see that I have added two options. 'protractor_install' and 'npm_install'. It is better to add an npm install before we are installing chrome driver and selenium just to make sure that, everything is ready beforehand.

  Now we will add the install task. I have added both 'npm_install' and 'protractor_install' to the install task.




Now, if we do a grunt install on the command console, it will show as everything uptodate, since we have already installed selenium and chromedriver in the current project folder.


PS: You can see some warning. Please ignore that for now. This is because we haven't added the README.md file and repo to the project, which we will add at the end.

To demonstrate the usefulness of what we did now, lets make a copy of our project folder. Make sure that, you are not copying the node_modules folder. I copied the files to 'exampleCopy' folder. Go to this copy folder in the command console.

Now, if we do the grunt install, everything should work, right? Let's see.



It is saying that a local copy of grunt is required. So, we have to do an npm install before we call any grunt tasks. Let's do that.

1. npm install

2. grunt install


Now you are ready to do the protractor tests. So, any other person using this repo only need to run two commands "npm install" and "grunt install" and they are ready to go.

Adding the HTML Reporter

When we run the tests, we can see that tests are passing or failing in the command console. What if we have hundreds of tests and also want to keep a record of the test results  whenever we run the tests? We need test reporters for that.

Protractor gives the options to add onPrepare function property on its configuration file. This onPrepare function will be called before each tests. Jasmine test framework(which we are using inside protractor) provides the option to add test reporter. We will get the metadata about each test by using these two configurations.

There is a node module available for generating screenshots for each tests. protractor-screenshot-reporter. However this doesn't process the meta data and gives us the result in a readable format.

I couldn't find a node module doing this for protractor. So, I created a module for generating this HTML report, on top of protractor-screenshot reporter, protractor-html-screenshot-reporter. What it does is generate an HTML report with all the details of the test like status, browser used, message, link to screenshot etc. (If you are finding some other modules which are doing the same, please add in the comments. )

Lets install protractor-html-screenshot-reporter module in our project.


We need to add this reporter in the Protractor configuration file.

1. require the protractor-html-screenshot-reporter module and assign to a variable.


2. add this reporter on the onPrepare function.


onPrepare: function() {
      // Add a reporter and store screenshots to `screnshots`:
      jasmine.getEnv().addReporter(new HtmlReporter({
         baseDirectory: 'screenshots'
      }));
   },

3. Run the test using grunt command. It will run the default task in the Gruntfile.



   We can see that the tests are passing. Lets see if the reports are generated.

Screenshots folder is created.


Inside the screenshots folder, one json and png with a guid filename is available. This is the metadata and screenshot respectively for the single test. There is also a combined.json which combines all the metadata into a single file and a reporter.html file which shows all the test reports.


Opening the reporter.html looks like below.



Clicking on View link opens the screenshot image also.




So, now you have an html reporter ready. But, there is one issue though. Whenever you run the tests, the reporter files are getting overridden. If I want to keep a track of tests, this is not going to help.

protractor-html-screenshot-reporter provides the option of pathBuilder function property to give your own dynamic paths. We will add this in Protractor configuration file.



  onPrepare: function() {
      // Add a reporter and store screenshots to `screnshots`:
      jasmine.getEnv().addReporter(new HtmlReporter({
         baseDirectory: 'screenshots',
         pathBuilder: function pathBuilder(spec, descriptions, results, capabilities) {
          
            var monthMap = {
              "1": "Jan",
              "2": "Feb",
              "3": "Mar",
              "4": "Apr",
              "5": "May",
              "6": "Jun",
              "7": "Jul",
              "8": "Aug",
              "9": "Sep",
              "10": "Oct",
              "11": "Nov",
              "12": "Dec"
            };

            var currentDate = new Date(),
                currentHoursIn24Hour = currentDate.getHours(),
                currentTimeInHours = currentHoursIn24Hour>12? currentHoursIn24Hour-12: currentHoursIn24Hour,
                totalDateString = currentDate.getDate()+'-'+ monthMap[currentDate.getMonth()]+ '-'+(currentDate.getYear()+1900) + 
                                      '-'+ currentTimeInHours+'h-' + currentDate.getMinutes()+'m';

            return path.join(totalDateString,capabilities.caps_.browserName, descriptions.join('-'));
         }
      }));
   },


I am just using a basic function to give the folder name as the current date and time and browser.

Note that I am using the path module for creating the file path (path is a native node module for handling and tranforming file paths. ). Make sure that, you have required the path and assigned to a variable.





Lets run the test again using grunt command in command console. As expected tests are running successfully.

If we check the project folder we can see that, the report is now generated in the path 'screenshots/22-Feb-2014-5h-58m/chrome'. Great. Now, on seeing the folder names itself, you know when the test was run and on which browser.


Finally, lets make this project ready to add as a git repo. 

1. It is not a good a good practice to push node_modules directory to the repo. In order to avoid node_modules to be pushed to git, we need to add it to .gitignore file. 

Create a new file in the root folder with name as .gitignore. 



We don't want screenshots and node_modules folder to be added in git. So, I have added both names in .gitignore. 

2. We will also add the repository details to package.json. 



3. Also, create a README.md file in the root folder and add some description of your project. 

Finally, we are ready to push to the repo. 

More Requirements

 Now that we are in a good state, we can check the further requirements.

  1. Generating tests based on style guide
  2. Doing CSS Regression testing

We will cover that in the coming articles. Stay tuned.

Please share your comments.

All the changes are added on github repo - protractor-e2e-bootstrap


35 comments:

  1. Reporting of month is off by One month(Dec(12) is reported as Nov). Reporting of OS is also wrong, Windows 8.1 is reported as XP

    ReplyDelete
    Replies
    1. Reporter gets the metadata using the browser.getCapabilities() method provided by protractor which internally calls webdriver.getCapabilities method. You can get more details by raising an issue in protractor repo.

      Delete
    2. Hi Jose,

      Thanks for the information. However is there any way to override the OS name? Or could you please direct me to right place where I can raise this issue?

      Thanks,
      Swarup

      Delete
  2. Replies
    1. Have you raised an issue on the github repo?

      Delete
  3. There is also an issue with the jasmine call in the function definition:
    ReferenceError: jasmine is not defined

    ReplyDelete
    Replies
    1. Have you raised an issue on the github repo?

      Delete
  4. We can avoid the reports overridden issue by parameterize the screen path along with the timestamp as below:

    var HtmlReporter = require('protractor-html-screenshot-reporter');
    var path = require('path');
    var currentdate = new Date();

    var screenpath = 'Reports/'+ currentdate.getDate() + "-" + (currentdate.getMonth()+1) + "-" + currentdate.getFullYear() + " @ "
    + currentdate.getHours() + currentdate.getMinutes() + currentdate.getSeconds();
    exports.config = {
    seleniumAddress: 'http://localhost:4444/wd/hub',
    specs: ['e2e/*.spec.js'],
    baseUrl: 'https://..../',
    jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000
    },
    onPrepare: function() {
    // Add a screenshot reporter:
    jasmine.getEnv().addReporter(new HtmlReporter({
    /*baseDirectory: function crDir(){
    var currentdate = new Date();
    var datetime = currentdate.getDate() + "-"
    + (currentdate.getMonth()+1) + "-"
    + currentdate.getFullYear() + " @ "
    + currentdate.getHours() + currentdate.getMinutes() +
    currentdate.getSeconds();
    var screenpath = 'screenshots'+ "_" + datetime ;

    return screenpath;
    },*/
    baseDirectory: screenpath,
    takeScreenShotsOnlyForFailedSpecs: true,
    pathBuilder: function pathBuilder(spec, descriptions, results, capabilities) {

    var monthMap = {
    "1": "Jan",
    "2": "Feb",
    "3": "Mar",
    "4": "Apr",
    "5": "May",
    "6": "Jun",
    "7": "Jul",
    "8": "Aug",
    "9": "Sep",
    "10": "Oct",
    "11": "Nov",
    "12": "Dec"
    };

    var currentDate = new Date(),
    currentHoursIn24Hour = currentDate.getHours(),
    currentTimeInHours = currentHoursIn24Hour>12? currentHoursIn24Hour-12: currentHoursIn24Hour,
    totalDateString = currentDate.getDate()+'-'+ monthMap[currentDate.getMonth()+1]+ '-'+(currentDate.getYear()+1900) +
    '-'+ currentTimeInHours+'h-' + currentDate.getMinutes()+'m';

    return path.join(totalDateString,capabilities.caps_.browserName, descriptions.join('-'));
    }
    }));
    },
    };

    ReplyDelete
  5. Nice information about the load testing!!! I prefer Loadrunner automation testing tool to validate the performance of software application/system under actual load. Loadrunner training institute in Chennai | Loadrunner course in Chennai

    ReplyDelete
  6. Jinto,

    I used the same configurations which u mentioned above but I haven't seen any report or screenshort generated for my scenario, also the folder i manually created for storing the reports get deleted when i ran my script. Please let me know ur thoughts on this!!

    ReplyDelete
  7. WebMethods Training in Chennai
    This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic..

    ReplyDelete
  8. There is a better way of doing protractor screenshots, have a look at https://github.com/azachar/protractor-screenshoter-plugin (disclaimer: I am the author)

    ReplyDelete
  9. There is a better way of doing protractor screenshots, have a look at https://github.com/azachar/protractor-screenshoter-plugin (disclaimer: I am the author)

    ReplyDelete
  10. Wow, the information on this site has helped ne to finish writing my thesis paper and I am grateful that I landed on it while I was looking for Urgent Help to Finish a Project. I will bookmark this site so that I can visit it occasionally to read more programming information. I wonder where I could have found such information if it was not posted on this site.

    ReplyDelete
  11. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  12. It is extraordinarily a magnificent work and the way in whatever u r sharing the experience information is excellent.Thanks for treating me to understand basic concepts.If want become learn for Java Training with OOPS knowledge to reach us Besant Technologies.To click the training details, Java Training in Chennai | Java Training Institute in Chennai

    ReplyDelete
  13. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  14. Nice Article, But I want to know that, there is any way to run that report.html file on browser automatically after creation of this file.

    ReplyDelete
  15. Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  16. really good piece of information, I had come to know about your site from my friend shubodh, kolkatta,i have read atleast nine posts of yours by now, and let me tell you, your site gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanks a lot once again.
    Hadoop Training in Chennai
    Big Data Training in Chennai
    Python Training in Chennai
    Python Training Centers in Chennai
    Data Science Training in Chennai
    Data Science Course in Chennai
    Data Analytics Training in Chennai
    Best AngularJS Training in Chennai
    AngularJS Training in Chennai
    QlikView Training in Chennai
    Informatica Training in Chennai

    ReplyDelete
  17. This is a good post. This post give truly quality information. I’m definitely going to look into it. Really very useful tips are provided here. thank you so much. Keep up the good works.
    Hadoop Training in Chennai
    Hadoop Training in Bangalore
    Big data training in tambaram
    Big data training in Sholinganallur

    ReplyDelete
  18. You got an extremely helpful website I actually have been here reading for regarding an hour. I’m an initiate and your success is incredibly a lot of a concept on behalf of me.
    Big data training in Marathahalli
    Big data training in btm
    Big data training in Rajajinagar
    Big data training in bangalore

    ReplyDelete
  19. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Devops training in velachery
    Devops training in annanagar
    DevOps online Training

    Devops Training in Chennai

    Devops Training in Bangalore

    ReplyDelete
  20. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  21. Thank you for benefiting from time to focus on this kind of, I feel firmly about it and also really like comprehending far more with this particular subject matter. In case doable, when you get know-how, is it possible to thoughts modernizing your site together with far more details? It’s extremely useful to me.
    python training in chennai
    python training in chennai
    python training in Bangalore

    ReplyDelete
  22. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    Python training in marathahalli
    Python training in pune

    ReplyDelete
  23. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 
    java training in chennai | java training in USA

    selenium training in chennai


    ReplyDelete
  24. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging.
    java training in chennai | java training in bangalore

    java online training | java training in pune

    ReplyDelete
  25. Some us know all relating to the compelling medium you present powerful steps on this blog and therefore strongly encourage contribution from other ones on this subject while our own child is truly discovering a great deal. Have fun with the remaining portion of the year.

    Data Science training in rajaji nagar | Data Science with Python training in chenni
    Data Science training in electronic city | Data Science training in USA
    Data science training in pune | Data science training in kalyan nagar

    ReplyDelete
  26. This comment has been removed by the author.

    ReplyDelete
  27. It has been simply incredibly generous with you to provide openly what exactly many individuals would’ve marketed for an eBook to end up making some cash for their end, primarily given that you could have tried it in the event you wanted.
    safety course in chennai

    ReplyDelete
  28. Just stumbled across your blog and was instantly amazed with all the useful information that is on it. Great post, just what i was looking for and i am looking forward to reading your other posts soon!

    angularjs-Training in tambaram

    angularjs-Training in sholinganallur

    angularjs-Training in velachery

    angularjs Training in bangalore

    angularjs Training in bangalore

    ReplyDelete