Loadrunner misses extra resources

Recently, I’ve encountered an issue with an AUT where it crashes VUgen during the generation stage of a recording session. Now I’m a big fan of record-edit-playback. I find the recording and generation logs hugely useful and I find that recording is the easiest and quickest way to get a view of the application.
I call it record-edit-playback because the recording isn’t enough on it’s own for me to use as a performing performance test. But as a foundation, it’s fine. And they rarely playback immediately in any case. Session ID’s are designed to prevent that kind of thing.

The logs in particular are great for helping you find and construct parameter-capturing code for session ids, and identifying user variables that you may want to emulate.

My process for recording is very simple though I haven’t documented it here before so I will now:

  • Record the script once
  • Save it as Scriptname_DATE_RAW
  • Save it again as Scriptname_DATE_WIP (work in progress).

That means you can always roll back if you need to. The WIP copy is the working document until it’s finished when it becomes Scriptname_DATE_FINAL. At that point I hive it off to the controller as well as my local repository. I don’t like cluttering up the controller with WIP versions. And I don’t like the controller pulling scripts across the network, I just think it’s poor practice.

But I digress.

As a solution to the fact I couldn’t record the script, I used Fiddler as a proxy to capture the urls I visited when manually executing the script in Firefox. Over on the DevDiary there’s an article about this, but the point I wanted to make was this. Loadrunner doesn’t capture everything that’s on a page. Fiddler output was about 40 lines for the homepage, a Loadrunner visit to just the homepage was capturing 10 lines of resources (I managed to get LR to do that before it died again).

It seems that if a resource (for example a .css) contains sub-resources Fiddler will see that but Loadrunner won’t. I don’t know if that is by accident or design. I don’t know if it’s implied that LR is geettng them but not explicit in the results and the logs. I intend to find out in due course but it makes me wonder how I’ve not seen this before in 15 years of performance testing. Maybe it’s specific to this project, I could believe that, we are uniquely complicated from what I’ve seen. But what if it’s not. How many issues could have been avoided if I’d seen a bottleneck on one of those resources – maybe an underperforming java-script for example? It’s all academic now anyway but it’s certainly something I’ll look out for in the future. And as an aside, maybe Loadrunners recording engine isn’t as good as I’ve always thought it to be? Interesting times. In Belgium…

Dev Diary

There’s a link to the dev diary over there on the right, currently amongst the links to hosts and wordpress themes. I’ll be adding more relevant content as I find it.

I’ve added a few days of diary regarding test stubs, vbs and vb6. It’s coming along nicely.

The dev diary will be much more informal that the main AS.Org site, this one is going to remain as my more professional aspect.

UPDATE:

The first working version of the Loadrunner Batch Schedule (Excel version) is now available on Dev.

Extending Loadrunner Scripts with C – Function Library #1.1

Actually, this is more like 1.1. In as much as it ties into the previous post. I was blogging about building audit logs and data files via an “audit” script. That’s what I call them, not sure if there’s a full blown technical name but I use them to verify, validate and build data to be used in actual test scripts.
So let’s suppose you have an array of data you’ve captured with web_reg_save_param (x,y,z,"ord=all",last); this is how to handle that code into an audit log.

vuser_init{

// write file header once

WriteToOutputFile(lr_eval_string(“card,psn,status”));

return 0;
}
The function as defined in the previous post.
int WriteToOutputFile(char * string)
{

char *filename = “c:\\gemalto_audit.txt”;
long file_streamer;

if ((file_streamer = fopen(filename, “a+”)) == NULL)
//open file in append mode
{
lr_error_message (“Cannot open %s”, filename);
return -1;
}

fprintf (file_streamer, “%s\n”, string);
fclose(file_streamer);
return 0;
}

And finally, the function in use…

action
{
char szParamName1[128];
char szParamName2[128];
char szParamName3[128];
...
// get number of matches from ord=all

nCount = atoi(lr_eval_string("{available_cards_psn_count}"));

//"available_cards_count" = 22 - boundaries are insufficiently unique
//"available_cards_psn_count" = 11
//"available_cards_status_count" = 22 - boundaries are insufficiently unique

for (i=1; i<=nCount; i++ )
{
j = i * 2;
sprintf(szParamName1, "{available_cards_%d}", j);
sprintf(szParamName2, "{available_cards_psn_%d}", i);
sprintf(szParamName3, "{available_cards_status_%d}", j);
strcpy(strToOutput,lr_eval_string(szParamName1));
strcat(strToOutput,",");
strcat(strToOutput,lr_eval_string(szParamName2));
strcat(strToOutput,",");
strcat(strToOutput,lr_eval_string(szParamName3));
WriteToOutputFile(strToOutput);
}
}

I find more often than anything else, capturing the data is easy enough, but getting at that data in a structured way in order to use it effectively at a later point can be painful. The above is a real-life example – Developers implementing the content management inconsistently meant that there was nothing uniquely identifying 2 of the fields I needed. If I tightened the left boundary or the right boundary, elements were missed.
I’m not criticizing developers per se, they can’t really be expected to think about a performance tester a year down the life-cycle of the project looking at source-code structure.
The workable solution was to capture the 11 values I needed for one element, the 22 value-pairs for the other elements, and just skip every other element in 2 of the arrays. Inelegant perhaps, it works though and I built that today so it may become beatified over time.

Extending Loadrunner Scripts with C – Function Library

So, I’m working at a new client, back doing the Loadrunner thing. One of the nice things about that is I get to re-use and refine code I’ve written previously for other clients. This article is going to contain some of these code snippets that I’ve used time and time again.

I’ve re-visited this code recently and found that a) it wasn’t very good, and b) I can do it better now – presented below is the better version. There will be an update inviting formatting etc.
And there’s no guarantee this is perfect.

Output to Text file

int WriteToOutputFile(char * string)
{

char *filename = "c:\\myfilename.txt";
long file_streamer;

if ((file_streamer = fopen(filename, "a+")) == NULL)
//open file in append mode
{
lr_error_message ("Cannot open %s", filename);
return -1;
}

fprintf (file_streamer, "%s\n", string);
fclose(file_streamer);
return 0;
}

Called like this:

WriteToOutputFile(lr_eval_string("bban_count: {bbanNumber_count} blah {bbanNumber_count}"));

I just added the function beneath vuser_init rather than creating a header file. For multiple vusers, it’s a good idea to parameterise the filename as they can’t all share. I recommend a vuser identification parameter as thats built in.
Or Timestamps for uniqueness.

I also have a dos script for joining them all back up again since I tend to use this function for creating custom audit logs to track test data states as it moves through a scenario / test cycle.

A failed web_find is not always an error


I’ve been working in Lithuania for the last 3 months. It’s cold over here. To keep warm (and paid), I’ve been writing some loadrunner scripts for a Scandinavian bank. One of them seeks to emulate a customer paying cash into their account. As you’d imagine, this is a high priority, high usage script so I’ve spent the last 3 weeks building this in the web protocol against a custom built CRM platform.
Continue reading

Automated Testing Best Practices (Basic)

I wanted to prepare an article that specifies the very basic best practices for Automated Testing, whether its functional or non-functional. It seems to me that all the tools, whether open-source or commercial allow for these essential practices, presumably because this is the way we should all be doing it… some of us aren’t, you know who you are and you’re very naughty. Also your testing isn’t as good as it should be…

This article is very much a work-in-progress as I braindump…

  1. All scripts should have a manual equivalent, because this is the foundation upon which the automation is built. It doesn’t have to be complicated, it doesn’t have to be as thorough as the automated script, but it does have to accurately specify the steps to follow.
  2. Some of the tools are best used in a record-edit-playback loop (Loadrunner for example). Others pretty much require you to edit them as you progress through your script, adding checkpoints and things as you go (QTP springs to mind). If its a pain to go and insert checks after the recording, that is just tough. Your test is only as good as the conditions it checks for.
  3. A good understanding of the tool is required before you begin any testing phase. Yes, we all have to start somewhere, and that somewhere should always be with training additional reading.
  4. All scripts have transactions, whether its a page click, a text entry, a mouseclick, whatever. Defining your test by these transactions makes debugging easier, makes the scripts more readable and ultimately makes you look professional. Alternatively, a multi-step business process should be mapped as a single logical transaction.
  5. All scripts should include checkpoints for every “transaction” regardless of whether there’s a condition defined. Again its not complicated but your checkpoints need to be sufficient that if something is broken AT ANY POINT your test can detect it.
    If you check for something unique to where you are in the AUT and set pass/fail criteria around that, that’s perfect. The sooner a test which has failed is known to have failed is the sooner the processor is moving on the next activity, saving time and money. And the sooner you’ll be debugging which is one of the fun bits in my opinion

First Steps with QTP part 3

I’ve recently been tasked with inserting data into an 20-year-old 16-bit application to give us a benchmark data set for an updated alternative. In standard investment banking terms, there are deals, to which orders are attached, and there are orders which can have multiple tickets.

An API had been developed for inserting Orders and Tickets but Deals were still being entered manually. I am averse to doing anything repetitive manually, which is how and why I ended up automating in the first place.

As every techie knows all user-generated data starts life in a spreadsheet. It might get stored in a database eventually but for input and updates, it’s a spreadsheet.
Looking at the tools available I had a choice between QTP and writing a bespoke API.
Looking at the data, and armed with the knowledge that QTP and Excel will talk to each other with minimal fuss, I thought it might be worth giving QTP a trial run.
Looking at the application, I figured “there’s no chance of making it work, but I’m paid to solve problems so spend half-a-day on it, and see what happens”.
Continue reading

First Steps with QTP

Matt:
So I’ve got around 10 years experience with Loadrunner, am well-versed with Winrunner, can code in half a dozen languages and still I never got around to learning QTP. Well now I have no choice since we’re using it at my latest client site.
The aim of this article is to provide guidance in QTP, how to start, what to look at, how to record a simple script and parameterise it. The essentials, basically.

I recorded a simple script going from Google to AS.org, inserted one checkpoint as I was doing so. It seems that the insertion of checkpoint is best done at record-time. I’m not clear yet on how to insert one afterwards…

My script looks like:
Browser("Google").Page("Google").WebEdit("q") _
.Set "automationsolutions.org"
Browser("Google").Page("Google").WebEdit("q").Submit
Browser("Google").Page("automationsolutions.org") _
.Link("Automation Solutions").Click
Browser("Google").Page("Automation Solutions").Check _
CheckPoint("Automation Solutions")

It’s very readable I suppose and handily is built around Visual Basic, one of the languages I am familiar with.

Standard stuff like Browser, Page, WebEdit, (“q”, by the way is the name of the textbox object on Google’s homepage. Check with Firebug if you want confirmation) Link and Click are container objects or Actions for specific data, but I don’t think “Google”, “Automation Solutions” or that CheckPoint tell me enough about what’s going on – is “Google” www.google.co.uk?, is “Automation Solutions” a link to the site or a string to be pushed to the browser? And what is that checkpoint checking for?

The Object Repository

Right-Clicking on the keyword “Google” brings up a menu with Object Properties uppermost in the list. Clicking on that brings up this:

!img_placeholder!

I’m not happy with the default properties and although QTP might feel they’re sufficient to identify the object, I want more information so I click the green +.
I can see URL is available for selection (as openurl) so I select that. It has no effect on the code, but in the event I need to debug, I can see how knowing the url could be useful.

!img_placeholder!

Right-clicking on the checkpoint brings up:

!img_placeholder!

It shows what text is being searched for, and it also shows that it can be parameterised. Having added it to the global table, I am forced to wonder how best to data-drive a QTP test.

Conclusion

Using google to search for a data string, find a link based on that string and then check that its on the page we expected might seem trivial but it’s allowed us to see the basics of QTP

First Steps with Selenium

Matt:
So, Selenium is an open-source toolset for testing web-based applications. It can be found here.
I started working with Selenium recently and I though it’d be an interesting post to document how I went from novice to competent user.

One of the interesting features of Selenium is that is can be used for performance testing. It’s my experience that performance testing is a costly business, so to see an open-source tool that can do it is rare.

Lets begin by looking at what Selenium is comprised of:

  • Selenium IDE – the scripting environment.
  • Selenium RC – allows further scripting and execution with variable data
  • Selenium Grid – Allows for multiple tests (or instances of the same test) to be executed.

It seems to me that the IDE and RC are doing the job of Loadrunners VUGen and Grid is handling the tasks of the controller. More on this in a future post.

At the time of writing, Selenium IDE is available as an add-on for the Firefox browser only, though the tests can be replayed (via RC) on Firefox, IE and Chrome.

Installation

Installation is made easy with the availability of 3 zipped packages (core, rc and grid) and the inclusion of the IDE as a firefox extension, (xpi) which will automatically install into firefox. The folks over at Selenium HQ also recommend getting an element inspector for your browser. As all the Selenium scripting I’ve done is in Firefox, Firebug is highly recommended, and freely available as an add-on or from http://getfirebug.com/ losartan medication.

Start by installing the IDE into firefox, and unpack the zip files somewhere sensible.

Hello Selenium!

Under the tools menu, there should now be an entry for Selenium IDE that leads us to this:

Selenium IDE

More to come, this article is going to be much bigger than I first thought…