Loadrunner misses extra resources

Recently, I’ve encountered an issue with an AUT where it crashes VUgen during the generation stage of a recording session. Now I’m a big fan of record-edit-playback. I find the recording and generation logs hugely useful and I find that recording is the easiest and quickest way to get a view of the application.
I call it record-edit-playback because the recording isn’t enough on it’s own for me to use as a performing performance test. But as a foundation, it’s fine. And they rarely playback immediately in any case. Session ID’s are designed to prevent that kind of thing.

The logs in particular are great for helping you find and construct parameter-capturing code for session ids, and identifying user variables that you may want to emulate.

My process for recording is very simple though I haven’t documented it here before so I will now:

  • Record the script once
  • Save it as Scriptname_DATE_RAW
  • Save it again as Scriptname_DATE_WIP (work in progress).

That means you can always roll back if you need to. The WIP copy is the working document until it’s finished when it becomes Scriptname_DATE_FINAL. At that point I hive it off to the controller as well as my local repository. I don’t like cluttering up the controller with WIP versions. And I don’t like the controller pulling scripts across the network, I just think it’s poor practice.

But I digress.

As a solution to the fact I couldn’t record the script, I used Fiddler as a proxy to capture the urls I visited when manually executing the script in Firefox. Over on the DevDiary there’s an article about this, but the point I wanted to make was this. Loadrunner doesn’t capture everything that’s on a page. Fiddler output was about 40 lines for the homepage, a Loadrunner visit to just the homepage was capturing 10 lines of resources (I managed to get LR to do that before it died again).

It seems that if a resource (for example a .css) contains sub-resources Fiddler will see that but Loadrunner won’t. I don’t know if that is by accident or design. I don’t know if it’s implied that LR is geettng them but not explicit in the results and the logs. I intend to find out in due course but it makes me wonder how I’ve not seen this before in 15 years of performance testing. Maybe it’s specific to this project, I could believe that, we are uniquely complicated from what I’ve seen. But what if it’s not. How many issues could have been avoided if I’d seen a bottleneck on one of those resources – maybe an underperforming java-script for example? It’s all academic now anyway but it’s certainly something I’ll look out for in the future. And as an aside, maybe Loadrunners recording engine isn’t as good as I’ve always thought it to be? Interesting times. In Belgium…

Dev Diary

There’s a link to the dev diary over there on the right, currently amongst the links to hosts and wordpress themes. I’ll be adding more relevant content as I find it.

I’ve added a few days of diary regarding test stubs, vbs and vb6. It’s coming along nicely.

The dev diary will be much more informal that the main AS.Org site, this one is going to remain as my more professional aspect.

UPDATE:

The first working version of the Loadrunner Batch Schedule (Excel version) is now available on Dev.

Loadrunner to interact with a VB6 Application as a test stub

So, following on from the previous article / rant…

I managed to re-build the excel spreadsheet & macro functionality within VB6. I’d provide more details but it’s largely bespoke and unlikely to aid anyone, including me, in the future.

The app makes a call to openssl in order to generate, encrypt and/or decrypt a user certificate.
It’s all called from the command line, and that was interesting to build and a pain to test.

In any case, for reference, to build an application in VB6 without any forms:

Go to project properties (for the entire project, not just a module) and select startup object as a sub – in my case SUB MAIN.

Define SUB MAIN as a standard module and either call all your code within it, or add you code within it directly.


Public Sub Main()

'Store command line arguments in this array
Dim sArgs() As String
Dim iLoop As Integer

'Assuming that the arguments passed from
'command line will have space in between,
'you can also use comma or otehr things...
sArgs = Split(Command$, " ")
For iLoop = 0 To UBound(sArgs)
'this will print the command line
'arguments that are passed from the command line
'MsgBox iLoop & "-" & sArgs(iLoop)
Next

'This is an example of the call, vb'd for testing
'Call MobileLogon("ECEEEJBUHEJUASDMCTBUFSFBEPIVEGAYDCFSDRAD", "pO5uDaWU", "C:\Temp\0057525949_____________UDID__EXCEL__TEST_MattLea0101.pem", "0057525949")

'This is the real thing.
Call MobileLogon(sArgs(0), sArgs(1), sArgs(2), sArgs(3))

End Sub

Testing is a pain because once you’ve made the executable, it only accepts the command line variables passed in (sArgs). So you end up putting a duplicated call in with hard-coded values – like the above.
And then you forget, make the executable and run it. It works – of course – because it’s using the hard coded variables.
And then you have to fire up the IDE, fix it, recompile, rebuild and retest.

And if you’re using loadrunner and the system command to test this, it can take a while:

The call:

strcpy(commandstring, lr_eval_string("C:\\Matt\\TestStubATE{vUserID}.exe "));
strcat(commandstring, lr_eval_string(" {sessionId}"));
strcat(commandstring, lr_eval_string(" {randomSalt}"));
strcat(commandstring, lr_eval_string(" {path_to_certificate}"));
strcat(commandstring, lr_eval_string(" {IngID}"));

lr_log_message(commandstring);

rc = system(commandstring);

This particular script uses a certificate generated by OpenSSL from Loadrunner. Only the certificate was missing a newline / carriage return, line feed. Easy enough to fix in code I thought. Well in C in Loadrunner it was a pain – seriously I spend half my life battling with string manipulations in C, why isn’t there a mid function, an instring, a substring function etc? And don’t get me started with StrTok.
So I built that in VBS:


Const ForReading=1
Const ForWriting=2

Set args = Wscript.Arguments 'Command line arguments

For Each arg In args
' Wscript.Echo arg 'For debugging
strFile = arg
Next

' Call edit_pem_file_workaround("0057525949_____________UDID__EXCEL__TEST_MattLea0101.pem")

Set objFSO = CreateObject("Scripting.FileSystemObject")
' folder = "c:\temp\"
'strFile = "0057525949_____________UDID__EXCEL__TEST_MattLea0101.pem"

filePath = strFile

Set myFile = objFSO.OpenTextFile(filePath, ForReading, True)
Set myTemp= objFSO.OpenTextFile(filePath & ".tmp", ForWriting, True)

Do While Not myFile.AtEndofStream
myLine = myFile.ReadLine

If InStr(myLine, "DEK-Info:") Then
myLine = myLine & vbCrLf
End If

myTemp.WriteLine myLine
Loop
myFile.Close
myTemp.Close
objFSO.DeleteFile(filePath)
objFSO.MoveFile filePath&".tmp", filePath


It looks for a line with DEK-Info on it and appends a new line onto it.
LR calls that in much the same way:

strcpy(commandstring, "C:\\Matt\\TestStub_Repo\\EditPemFile.vbs ");
strcat(commandstring, buffer);
system(commandstring);

Buffer contains the path to the certificate file.

So 2 test stubs to workaround issues identified, no great problem there. Except the vb6 one is an executable, and I have no idea how threadsafe or reliable that’s going to be. It’s command line only so the memory and cpu footprint is minimal but better safe than sorry. Especially when performance testing so I created another VBS to copy the teststubAte executable and rename it an teststubAteN.exe where N will represent the vUser ID within the injector.
I find out tomorrow if that works when my POC for this project is built.

Is it sad that I’ve enjoyed the complexity of all this? It’s not. Is it going to be annoying when someone tells me it was all unnecessary, that I could have done it another better way. Well, yes, it will be. IF it happens.

Extending Loadrunner Scripts with C – Function Library #1.1

Actually, this is more like 1.1. In as much as it ties into the previous post. I was blogging about building audit logs and data files via an “audit” script. That’s what I call them, not sure if there’s a full blown technical name but I use them to verify, validate and build data to be used in actual test scripts.
So let’s suppose you have an array of data you’ve captured with web_reg_save_param (x,y,z,"ord=all",last); this is how to handle that code into an audit log.

vuser_init{

// write file header once

WriteToOutputFile(lr_eval_string(“card,psn,status”));

return 0;
}
The function as defined in the previous post.
int WriteToOutputFile(char * string)
{

char *filename = “c:\\gemalto_audit.txt”;
long file_streamer;

if ((file_streamer = fopen(filename, “a+”)) == NULL)
//open file in append mode
{
lr_error_message (“Cannot open %s”, filename);
return -1;
}

fprintf (file_streamer, “%s\n”, string);
fclose(file_streamer);
return 0;
}

And finally, the function in use…

action
{
char szParamName1[128];
char szParamName2[128];
char szParamName3[128];
...
// get number of matches from ord=all

nCount = atoi(lr_eval_string("{available_cards_psn_count}"));

//"available_cards_count" = 22 - boundaries are insufficiently unique
//"available_cards_psn_count" = 11
//"available_cards_status_count" = 22 - boundaries are insufficiently unique

for (i=1; i<=nCount; i++ )
{
j = i * 2;
sprintf(szParamName1, "{available_cards_%d}", j);
sprintf(szParamName2, "{available_cards_psn_%d}", i);
sprintf(szParamName3, "{available_cards_status_%d}", j);
strcpy(strToOutput,lr_eval_string(szParamName1));
strcat(strToOutput,",");
strcat(strToOutput,lr_eval_string(szParamName2));
strcat(strToOutput,",");
strcat(strToOutput,lr_eval_string(szParamName3));
WriteToOutputFile(strToOutput);
}
}

I find more often than anything else, capturing the data is easy enough, but getting at that data in a structured way in order to use it effectively at a later point can be painful. The above is a real-life example – Developers implementing the content management inconsistently meant that there was nothing uniquely identifying 2 of the fields I needed. If I tightened the left boundary or the right boundary, elements were missed.
I’m not criticizing developers per se, they can’t really be expected to think about a performance tester a year down the life-cycle of the project looking at source-code structure.
The workable solution was to capture the 11 values I needed for one element, the 22 value-pairs for the other elements, and just skip every other element in 2 of the arrays. Inelegant perhaps, it works though and I built that today so it may become beatified over time.

Loadrunner with Excel, DLLs and Exe’s, also a Little bit of test stubbing

So, I’ve spent the last couple of weeks trying in vain to make Loadrunner interact with OpenSSL.
I managed to get the DLL declared, and recognised and was able to turn 50% of the existing VBA code into operational C.
This took a long time because partly my knowledge of the process is not where it should be, but then I reached a point where the VBA passes a byte array as an input to a function.
My Loadrunner doesn’t recognise bytes, just about understands arrays and certainly wasn’t about to do this.

So… I figured well, it’s in excel anyway lets just have LR launch the excel process, grab the data it needs and get out again. TOO SLOW. As I knew it would be.

So now I’m rebuilding it as a standalone VB app. I have no idea if this will work, let alone if it will work quickly but I’m running out of ideas so here we are.

Ultimately, this is stage 2 of an interaction upon a webservice, whereby the user (actually the users phone) says HELLO, and then performs a LOGIN.

HELLO or MobileHello to give it the full name of the webservice is just a call to a server and an xml response. Loadrunner can fire the url at the server and read the response.
Bits of that response, session identifiers essentially, are reused in the LOGIN with encryption based upon public and private keys and certificates.

My task is to emulate MobileLogin following a successful HELLO, I’ve captured the values and now I need LR to fire against a VB executable with the parameters it has captured, perform the encryption and derive the next url. It then needs to fire against that url to exercise the server. I’ll make it work, but this has proven to be one of the more difficult tasks I’ve dealt with.

still procrastinating?

Actually no. I’m working out of Brussels, Belgium. And I’ve found my internet access outside office hours to be incredibly flaky. I’ll find a better hotel or buy a damn router myself if it continues like this.

I have so much to do and weirdly am being prevented from doing it by internet speeds beneath 56k modem equivalents. This is extremely annoying but add on the current hotels scheme whereby you may have ONLY one active connection.

As a techie, I have 2 smart-phones (1 UK, 1 Belgian Burner), A kindle, 2 Laptops and an iPod. (The sheer weight of chargers I waddle around with is ludicrous). One active connection is for Luddites. I’d kind of expect this kind of thing in a (let’s say) less technical nation, but Belgium is central europe, the home of the EU and Nato, hardly a country stuck in the dark ages.

Also, and this is merely by way of an excuse, they have beer. Lots of beer. So, I’ll be back, updating as and when, once my connection is sorted. I do have a deliverable created. And some loadrunner tricks to update – one of my Belgian projects is to recreate a visual basic program in C. Lots of string manipulations. Ultimately it failed, as Loadrunner won’t create the byte array OpenSSL needs as an input. I may be able to fix that with a header file or I may go a different route. That’s Mondays job.

Oh, yeah, the deliverable is a data-generator which reads/ writes a dat file for a loadrunner script.

Hello world!

and so it begins…

This will be the development diary for all things Automation Solutions – I have much to do just to make it look like my site. I’ll be making images and picking themes

Then we’ll have some posts

For example: I have 2 projects in mind – the Loadrunner batch scheduler (multi-version), the Execution scheduler (which is just a booking system with a fancy name) and there will also be snippets of code in multiple languages as I get up to speed in perl, ruby, java et al.

 

 

And then there were 2…

I’ve just been asked by my boss to build a proof of concept for a scheduling system, not to execute performance tests but to book time slots on the shared controller.
It’s not unusual that a controller would be shared by a number of testers, very few projects require and use 24.7 access to Loadrunner, most scripting is done on an independent machine (VUGen can be installed and run independantly of the Controller) and there are often multiple workstreams.

Now I’ve worked in a million different places, and I’ve seen some booking solutions over the years and it is my opinion that most automated testers can’t be doing with them, they’re clunky, slow to complete and just another layer of irritating bureacracy. Mine will no doubt be the same.

Looking at 2 possibilities off the top of my head:
1. Spreadsheet on a shared drive. This has the advantage of being simple to build, but sharing requires opening and closing the document to prevent locking, and there’s usually one hog in my experience.
2. Online booking system with a php-based calendar and a webform tied into a mysql database at the back-end. I can build that with resources acquired on the net and customise to fit but it will still take longer than the spreadsheet.

As a POC-request, I’ll end up doing both and asking the testers which they prefer, and the answer will almost certainly be “Neither, can’t we just get together and figure out who needs it when they need it.”
Yes, yes we can. Over a beer?

IT doesn’t rain but it pours…

Further to my last post, on the vague subject of a test run database and results repository. There are, of course, additional features to add to that. Especially if you wanted to provide it to clients at a cost.

It would need to be able to cross-match results on the fly, comparing like-for-like transactions/scripts/scenarios.

I’m not proposing to re-develop the analysis tool for loadrunner, that’s outside my scope at the moment (probably 😉 )
But something comparing Run A vs Run B of the same test in pure numerical terms sounds like a job for perl to me.
And yeah, maybe have it draw a graph, pretty sure that’s do-able on the fly.

So Working, Building and Researching. What’s new?