Ok, I’ll hold my hands up, I’ve been a bit remiss with the updates and the projects and the ra,ra,ra. In my defence I’ve been busy working for a living and learning some interesting new things, not least Apache JMeter.
I knew JMeter a little bit from previous projects, but my latest assignment is 100% JMeter for performance testing so that’s been a steep learning curve (although being paid to learn doesn’t hurt either).
In any case, I’ve gotten scripting covered and modularised test fragments have been produced. That was surprisingly easy, partly because it was handled by other people beside myself and I’m looking to leverage their automated tests to form a large part of the performance test pack, and partly because much of the testing is submitting XML messages to specified endpoints and once you have the basic mechanism covered, it essentially a copy + paste job with some small adjustments (usually the xml payload, which may or may not be parameterised).
You may be asking so what’s with todays post then, Matt?, and I shall tell you now.
A jmeter script / scenario – “.jmx” file can take variables from the command line and that is the preferred approach for most people since they’re using small sets of scripts, typically one, or re-using variables or running from the gui. The performance test pack on the other hand consists of multiple scripts within the jmx, with all the usual runtime settings necessary needing to be provided in one way or another. At the moment, the pack contains ~20 different scripts, with their attendant calls to datafiles etc.
I could hard-code the vusers, the pacing, the duration, the ramp-up settings, the thinktime, the paths to input data and output data, the reports directory etc.
I could prepare the command line request to call all 30 scripts with those settings but I can practically guarantee an errant space or a misplaced -j and then a test would be ruined, data burned, environments would need resetting, civilisations would fall, economies fail and life would probably cease on this planet. Or, an hour of time may be wasted. Definitely one of those.
So I want to create an interface to the batch file – and this is where I’m annotating that desire and figuring how much effort that’s going to be.
Within each jmeter call, I can setup the variables placeholder, starting with threadgroup – Number of threads = Number of vUsers. If I set each group to 0 by default then within the call, I can activate only the groups (scripts) necessary.
I should add that along with the 20 scripts for the major system of records (which is actually 2 applications sharing a database), there are additional stubs, interfaces and applications which need to be performance tested, potentially simultaneously (actually, definitely simultaneously, I was just underplaying the complexity 🙂 ).
Demo.JMX and Demo1.bat
So, to begin with, I knocked up the tests, fragments etc. That process will form another article at some point in the future, all about JMeter, but it’s going to cover the basics and I don’t have the time to revisit that at the moment.
The jmx is documented in the images below. It only has 2 fragments as a proof of concept, login and logout.
Alongside the jmx file is a batch file which is used to launch the jmx with it’s runtime settings declared.
@echo off
cls
setlocal
REM cd /d %~dp0
pushd %~dp0
Call ..\..\apache-jmeter-2.11\bin\jmeter -n ^
-t demo1.jmx ^
-Jlogin=su ^
-Jpassword=gw ^
-Jthreads=5 ^
-Jramp=0 ^
-Jduration=120 ^
-Jserver=qa2.bolt.admiral.uk ^
-Jport=8081 ^
-Jprotocol=http ^
-Jpath=/pc/PolicyCenter.do
PAUSE
I found that you can either have declared iterations or duration, not both, and additionally it doesn’t appear to be possible to declare them both, populate only one and ignore the other. It just didn’t work for me at all. The investigation continues but for now, I’m going to use duration, setting it to a low-level when I need single(ish) iterations.
This works adequately / perfectly well depending on whether duration is an acceptable or perfect solution for you. And I should add there’s no reason why I couldn’t create a second demo.jmx with the iterations populated. I could even have the batch file decide on that conditional which jmx to execute.
(Maybe later because…)
The issue isn’t with that, it works fine for me, but when I look at the full test, I find I have this:
@echo off
cls
setlocal
REM cd /d %~dp0
pushd %~dp0
Call ..\..\apache-jmeter-2.11\bin\jmeter -n ^
-t ModularTest.jmx ^
-Jlogin=su ^
-Jpassword=gw ^
-Jthreads1=5 ^
-Jthreads2=5 ^
-Jthreads3=5 ^
-Jthreads4=5 ^
-Jthreads5=5 ^
-Jthreads6=5 ^
-Jthreads7=5 ^
-Jthreads8=5 ^
-Jthreads9=5 ^
-Jthreads10=5 ^
-Jthreads11=5 ^
-Jthreads12=5 ^
-Jthreads13=5 ^
-Jthreads14=5 ^
-Jthreads15=5 ^
-Jthreads16=5 ^
-Jthreads17=5 ^
-Jthreads18=5 ^
-Jthreads19=5 ^
-Jthreads20=5 ^
-Jthreads21=5 ^
-Jramp1=0 ^
-Jramp2=0 ^
-Jramp3=0 ^
-Jramp4=0 ^
-Jramp5=0 ^
-Jramp6=0 ^
-Jramp7=0 ^
-Jramp8=0 ^
-Jramp9=0 ^
-Jramp10=0 ^
-Jramp11=0 ^
-Jramp12=0 ^
-Jramp13=0 ^
-Jramp14=0 ^
-Jramp15=0 ^
-Jramp16=0 ^
-Jramp17=0 ^
-Jramp18=0 ^
-Jramp19=0 ^
-Jramp20=0 ^
-Jramp21=0 ^
-Jduration=120 ^
-Jserver=qa2.bolt.admiral.uk ^
-Jport=8081 ^
-Jprotocol=http ^
-Jpath=/pc/PolicyCenter.do
PAUSE
And that’s just for the one system, there are currently 8 others, so I have to figure out a way around this now, and I’m thinking:
a) Simple – build a dashboard in excel which allows the batch file to be constructed and executed on the fly. For all systems I could have a tab and a dashboard and a single execute button. And I know this could work (because I’ve built part of it already but …)
b) Complex – build a gui-based front-end dashboard which will construct and execute the batch files.
For each system, there will need to be a seperate bat file to call a seperate jmx file (the directory structure is identical but each has it’s own root directory as per the picture above, otherwise I could centralise the whole thing). Calling multiple bat files from a bat file is the easier workaround incidentally but I like GUI’s and it’ll look entirely more professional.
A colleague of mine has created a simple interface in Eclipse to do this for a single test so I’m going to leverage that and extend.