Code base clean-up with a lot of dead code removal
CLIF core:
New "shared code server" mode enabling for multiple, independent, simultaneous test deployments sharing a common code server (automatically launched by the CLIF registry)
Some fixes and improvements of the command line interface, including a new feature for waiting for the registry to be ready
Important bug fix with regard to global state management, which solved a deadlock threat in test plans with several load injectors and/or probes
Fixed problem with multi-lines results in action reports file
Completion of the embedded smart network configuration utility
Eclipse GUI: miscellaneous fixes and improvements, including management of new clif.opts file
Swing GUI: fixes and improvements in the embedded reporting tool
Full revision and upgrade of the CLIF plug-in for Jenkins:
full upgrade with regard to Jenkins server
no longer based on CLIF's ant-based interface but on the new shell/batch script interface
full revision of parameters and management of ProActive or legacy versions of CLIF
some improvements and fixes in the report generator
ISAC plug-ins:
new SvnInjector for load testing an SVN repository
new GitInjector for load testing a GIT repository
new FileHandler plug-in bringing miscellaneous file-related facilities
full revision and upgrade of the LdapInjector
HttpInjector enhancements with regard to cookie management and response codes management
Fixed primitives wasNotified and wasNotifiedN in commons' Synchro plug-in
New not_test_and_test condition in Counter plug-in
Now the mavenization branch produces full-fledged ProActive CLIF distributions: CLIF server, Swing GUI, plug-in for Jenkins
Merged with all CLIF's trunk evolutions listed above
Includes a prototype feature for deploying CLIF test plans, and possibly distributed applications under test, using a ProActive Workflows and Scheduling platform
Development of a new command line tool based on a plain shell/batch script instead of Apache ant, introducing a new configuration file management enabling sharing a CLIF runtime among an arbitrary number of test projects with different configurations.
Development of a new configuration facility to transparently choose the right network addresses for the CLIF infrastructure.
ISAC plug-ins:
Enhanced "deep load" feature in HttpInjector: now includes CSS and JS, and gives the choice of what kind of element to load.
New primitives for FileReader, enabling loading files and clearing contents
New TcpInjector, an enriched version of the SocketInjector
Progress on the mavenization branch:
creation of artifact clif-api, containing API definitions moved from clif-core;
Decision and actions to make the ProActive branch the reference for the maven build chain;
Prepare for moving the CLIF plug-in for Jenkins to the Jenkins project official repository;
Merge with trunk to get all new features and further reorganization.
CLIF in a container: successful experiments to deploy CLIF nodes and ProActive CLIF nodes as Docker containers.
Communication:
talk at OW2 conference about using CLIF for testing Orange's Forge based on Tuleap ALM (see the presentation video and slides)
talk at Paris Open Source Summit about CLIF as a Service for performance testing in continuous integration over OpenStack
Results for year 2014
New production version 2.2.1, with binary distributions for MacOSX, Windows and Linux
the system probes implementation, based on the LeWYS project, has been fully replaced with a brand new implementation based on the SIGAR open source project. This change is absolutely transparent to CLIF users, who are still able to use their existing test plans with CPU, memory, disk and network probes, but now covering many more OS and architectures (refer to SIGAR documentation from Hyperic support)
further efforts to improve user experience with the main (Eclipse-based) CLIF console, including contextual help in the test plan editor, streamlined error reporting and an important patch solving a big instability problem
new command-line facilities to get help with probe parameters
first release of the embedded performance analysis and reporting tool
introducing .bat scripts for Windows to be used as a replacement to some commands of the ant-based command-line interface (to be extended and generalized)
work on ISAC plug-ins: improved CommandLineInjector, Counter and FileWriter plug-ins
Work on the Jenkins plug-in: fixed error rate in performance reports and improved properties management for ProActive CLIF
From a source code maintenance perspective:
maintenance of the ProActive branch
progress on the mavenization branch (still not fully functional)
Resulting from our involvement in the OpenCloudware collaborative project, a CLIF-as-a-Service architecture was built and demonstrated to automate performance testing, possibly in a continuous integration process.
CLIFaaS includes the deployment and configuration of the distributed load injection system, and possibly the deployment and configuration of the distributed application under test.
See the presentation slides and the demonstration video.
Results for year 2013
production version 2.1.1 was released at the beginning of the year, with binary distributions for MacOSX, Windows and Linux
work on ISAC plug-ins:
added new features to plug-ins Common, Counter and HttpInjector
new FileWriter plug-in to write arbitrary contents to arbitrary files during test execution
minor fixes and improvements on CLIF core, mostly on the "quick statistical reports"
miscellaneous fixes and improvements to the CLIF plug-in for Jenkins
from a source code maintenance perspective:
maintenance of the ProActive branch
creation of a "mavenization" branch dedicated to a full reorganization of the code base to support a maven-based build process (to be completed)
communication on "CLIF meets Jenkins" at OW2con'2013, dealing with the benefits of using CLIF with a continuous integration server
real-scale test-bed of CLIF as a Service on a community cloud (injecting traffic from 20 PCs of coworkers distributed through a whole country)
Results for year 2012
enhancement of the CLIF plug-in for Jenkins continuous integration server
wizard for importing a full CLIF test project with a single click!
improved automatic reporting
brand new, full-fledged, DNS load injector, including DNSSEC support and a dedicated data set provider
new "Quick Stats" feature: per-request type statistics, aggregated through all load injectors, generated as a single CSV-formatted file once measures are collected
debug and improvement of the interactive analysis and reporting tool
new SelfBench tool box: controller components providing smart benchmarking features (for advanced users)
automatic saturation detection (SaturationController) according to threshold policies on response times or resource usage
self-driven load injection (InjectionController) according to a queuing model-based regulation algorithm
further work on the SVN branch based on the ProActive middleware, applied to community cloud-based load testing with CLIF.
This work was carried out by Orange Labs (CLIF project) and ActiveEon (ProActive project), and was presented at OW2's Annual Conference in November.
This work won the Special Prize from OW2's Technical Committee. See the Community CLIF presentation
background research work: CLIF execution on cloud computing/virtualized infrastructures, PhD on self-optimization (in French) with CLIF+SelfBench
Results for year 2011
beta version of the new interactive analysis and reporting tool available from the SVN trunk
new production release 2.0.7, coming with:
extended Synchro ISAC plug-in for synchronization between distributed test scenarios
optimized and enhanced deployment to support large scale test plans: more than 1000 distributed load injectors (with an average deployment time of 100ms per load injector), support for partial deployment, deployment time-out
new development branch based on the ProActive middleware to bypass some networking issues (routing, firewalls...) and provide CLIF as a service on a community cloud
