My first impression of the Agile process is that it is very progressive. I can easily see the progress made by each developer as well as the entire team on a daily basis. This was rarely the case in the previous waterfall development environment. The several initial releases of this project have confirmed my observation that being able to deliver early results, with much sooner return on investment if you will, is one of the Agile method’s greatest strengths.

The Agile process is light-weight as opposed to the traditional well-structured heavier waterfall process. Light-weight does not mean that there is less control or planning. My understanding is that the Agile process distributes the planning and control of uncertainties to each team member in many small steps over a relatively short period of time, avoiding the big long term visionary statements and aggressive targets often set during the waterfall process. Agile planning is focused more on individual estimates over a short period of time rather than a lengthy planning document covering a much longer period. The Agile process requires constant revisiting of plans and schedules. This benefits the developers and maintains clarity and focus. Individuals and interactions are the keys to build a common understanding of the project goals and the ways to reach them.

Changes of project requirements are often a nightmare to developers in the waterfall process because they have to rewrite their functional specification and design documents. They may have to re-architect their software implementation. All of these updates require re-review and re-approval. The Agile process, on the other hand, encourages and embraces the changing requirements. Product owners constantly give feedback to developers based on their discussion with customers. Developers are more adaptable to the changing UACs (user acceptance criteria). Because Agile uses iteration-based scheduling (versus release-based scheduling) and because a single iteration is much shorter than one release, any significant changes in UAC can be reflected in the new stories for the next iteration.

During the iteration planning and release planning phases, software developers, have to work very closely with product owners and other stakeholders. It is absolutely crucial for every one of the developers to be involved in this phase because in order for the developers to solve the issues, we must first fully understand them. This is much harder in reality than it sounds. The Agile process encourages individual accountability, self-organization as well as collaboration between stakeholders. In practice, we use daily stand-up meetings which include accomplishments and obstacles for that day and a list of the things to do the next day. In addition routine communications occur between team members including all stakeholders in the project. This prevents any hidden issues and encourages team members to respond. The Agile process emphasizes working software over cumbersome documentation. As soon as we check in source code and mark a development task as being completed, the QA team is able to test it without any delay. This guarantees that we deliver completely developed and fully tested features in a much shorter period of time.

No matter how well the Agile process is defined, a project’s success depends to a large extent on how well you follow the process. Understated documentation, such as user story or use case related documentation, could lead to confusion especially for a larger project. Team members tend to rely heavily on email exchanges and wiki pages to record design thoughts and discussions without any version controls. Team members’ flexibility in accepting the changing UACs is one of Agile’s strengths. However, changing UACs too frequently or too late in the iteration will cause a lot of frustration among developers.

As of now, my knowledge and experience with the Agile process is still very limited. Even without much formal training or many years of Agile experience, I am already excited with and quite encouraged by a lot of benefits the Agile process has offered to us.

Looking back at past test automation projects that MapleWorks has been involved with leads me to conclude that test automation is no different than any other software development project.

Typically test automation tools are built using scripting or programming languages; a commercial platform such as AutomatedQA’s TestComplete or Emprix’s Hammer; an open source platform such as STAF; or an in-house development platform. Regardless of the approach taken, test automation is a product. It is a stand alone tool used to trigger events and then report whether the event triggered was handled appropriately by the system under test.

The key attributes surrounding test automation are:

• Understanding how the system under test operates
• Understanding the characteristics of the automation test tool
• Defining a precondition
• Creating an automated test
• Verifying the test case outcome
• Removing all evidence about the test

Understanding how the system under test operates is the most important component of any test automation activity. Knowing what triggers need to be injected into the system is crucial since the core component of any automated test case is to create activity on the system under test. Understanding what output is generated by the system under test is equally important because this output will be used by the test automation platform to declare whether the automated test case passes or fails.

Understanding the characteristics of an automation test tool is crucial in the construction of an action (the automated test case) from the desired intent (wanting to test the system). Each test tool is bound to some technology and each technology has limitations. The resourceful test case developer has the ability to use creativity and possibly out of the box approaches to bridge the gap between what is needed to trigger activity on the system under test and how to manipulate the test tool to generate this trigger. Understanding the automation test tool is also important in defining mechanisms for retrieving output data from the system under test.

A precondition essentially states what the starting point is prior to injecting the trigger onto the system under test. This precondition needs to be applied to the system under test as well as the automation test tool. A state machine analogy can be applied to this idea. In order to arrive at state “B”, event “B” must be applied from state “A”. Precondition requires that the system under test and the test tool be initialized and set to state “A”.

Creating an automated test case is the mechanics of applying the lessons learned from the previous two points in the construction of a tool to test the system under test. The test tool identifies the language used to create the test case and the decision making criteria in evaluating the test case outcome.

Verifying the outcome is a means to determine if the system under test is sane and behaving as expected. Once all the triggers are applied it is important to retrieve information from the system under test in order to evaluate its behavior. The automated test tool needs to obtain this data, and determine if the data meets expectations. If the expectations are satisfied then the test succeeded. If the expectations are not satisfied then the test case failed.

A failed test case is just is as valuable as a successful outcome because this suggests that the system under test is faulty.

Removing all evidence is an eye catching phrase to describe the post condition cleanup step. The goal is to place the system under test as well as the automation test tool into a known state so the next automated test case can be run. As stated previously, the test tool and system under test needs to be reset to state “A”.

At the end of the day successful test automation requires:

• Understand and prioritize your requirements

Understand the product that needs to be tested

Understand how the intended user is planning on using the tool

Understand how the system under test reports a successful or failed outcome

• Gain knowledge

Prototype the most complex procedure the intended user wants tested

Understand the technological limitations imposed by the system under test

Understand the limitations imposed by third party tools

• Go off and build the automation tool.

Design a suitable architecture

Implement the solution

The convergence of data, voice and video poses increased challenges for testing, as the network has to perform to the user’s expectations across different media types. The challenges in this area are two fold. One is to achieve a very high coverage of scenarios in a time, cost effective, and efficient manner. The other challenge is to achieve very high coverage of scenarios, the amount of equipment needed can grow to a point where it would be cost ineffective, or physically unfeasible to create sufficiently complex environments. Therefore it is important to build expertise in the following key areas:

• Automation

· Explore innovative and cost effective means of tying systems together to operate in a coordinated fashion to achieve test scenarios

· Leverage techniques of massively parallel programming and grid computing to achieve faster test scenario execution

· Develop adapters for commonly used and cutting edge test equipment, so that it can be controlled through an automation framework

• Simulation

· Create generic simulation frameworks that can be adapted to any kind of specific network infrastructures and protocols

· Integrate simulation frameworks with a computing grid to allow simulations to use variable amount of CPU power

Network management and support systems are needed for all three layers in the NGN.  Management of infrastructure in an Enterprise or Carrier environment is getting more complex as data, voice and video converge into one ubiquitous IP packet based network.  The challenge comes from migrating legacy equipment with next generation infrastructure to work harmoniously to provide a high quality of service to its users.  Providers today are already seeing the commoditization of their services by increased competition.  Therefore next generation management systems need to manage more, reduce costs, and be agile to meet further business needs.

Key areas to focus on are:

• Understanding trends in management protocols like NETCONF and WS-Management
• Using cutting edge Rich Internet Application technologies to enable interfaces to the web without compromising usability or functionality that customers have grown accustomed to
• Developing new methods for fast data store and retrieval to make data collection more scalable
• Devise new algorithms to ensure QoS is maintained across different services
• Devise new methods for quicker fault detection and fault resolution
• Create ways of making the management of devices as generic as possible so that systems can support next generation infrastructure

Call Control is one of the most important parts in the control layer of the NGN.  Signaling is used in the network to set up calls between two end users and to pass information about the call state between the various call agents involved in handling the call.  IMS uses Session Initiation Protocol (SIP) to create a signaling network over the IP network.  The Real-time Protocol (RTP) is used for media transport

Research and development work in the following areas is very important.

• Protocol stack development and analysis
• System high availability
• System scalability
• Network security

The open source community is involved in some important areas. Particularly in integrated access devices and IP PBX software projects.  An example is the Freeswitch project.

The next-generation network is a high speed packet or cell based network which can provide multiple services such as voice, data, video and multimedia. It has a high bandwidth multi-service core and intelligent edge. It provides quick service delivery and provisioning. Next-generation networks are based on internet technologies including Internet Protocol (IP), Multiprotocol Label Switching (MPLS), and Session Initiation Protocol (SIP).

Network convergence is one of the key subjects in the next-generation network. The goal is to bring together voice, data and video at the network level, at the system level, at the application level, and at the device level. Convergence in transport means all traffic sharing a common packet-based network which is usually IP based. At the system level, the softswitch will replace the traditional circuit switch. It enables voice communication over the IP backbone. IP PBX and multi-service devices enable the enterprise to access the network easily and reduce cost. More and more applications will be developed to use the new infrastructure. These new applications integrate voice, data and video at the desktop as well as in the server. Examples of this include multimedia conferencing, multimedia messaging etc.

The IP Multimedia Subsystem (IMS) is a standardized NGN architecture originally developed by the 3G Partnership Project (3GPP). It defines a generic architecture that offers VoIP and other multimedia services within wireless and wireline infrastructures. The IMS architecture is divided into three layers: service (application), control and transport (access). Following diagram is a simplified view of the layered architecture of IMS. Major vendors like Ericsson, Lucent, Nortel and Siemens have made substantial investment in IMS software. There are a lot of challenges and also a lot of opportunities in this technology evolution process.

To expedite development and quality assurance cycles, many software projects incorporate automated testing. Automation of testing can greatly reduce phase times and reduce costs for the any outsourced or internal project. By analyzing the way test automation is done, Mapleworks has developed a solution called Mapleworks Automated Testing System or MATS.

What is required from a test automation system?

1. The ability to run different types of tests:
   • Unit testing
   • Full system testing
   • Integration testing
   • Regression testing
   • Performance testing
2. Consistency and reliability
3. Observable detailed results with comparison ability
4. Flexibility – test and test suites

At Mapleworks we’ve found some practical problems with test automation. Take the case of a long and involved test script. These types of scripts can be recorded by a test platform or created manually with a sustained effort. If an essential piece of functionality has to be modified, then the script must be re-recorded or re-factored in some way. And say this modification is applicable to several different places in the delivery, the cost rises exponentially to the quality assurance cycles.

At Mapleworks, we have designed a novel approach to solving this problem. Using the TestComplete platform, we have added an abstraction layer that takes an arbitrary label and points it to each testable object. This de-couples the test script and testing engine from the tested program.

What we can now do is record an intricate test procedure, convert it to a MATS (Mapleworks Automation Test System) script and create the mapping file for the testable components.

An added benefit from this architecture is that the QA team can write tests before the project is completed. All that is required to run testing suites is the mapping file to be filled in with the actual component internal name which can be obtained using the TestComplete ‘map object from screen’ tool.

The mapping syntax is straightforward: an arbitrary descriptive name, its object type (button, list, etc), and its component internal name which is implementation specific.

Sample Component Definition File:

Template Editor|Catalogs, Tab,.WinFormsObject(’WindowDockingArea’, ”, 3).Pane.Panes.item[1]

Template Editor|Catalogs|=Catalogs|,Copy,WinFormsObject(’TemplateEditorFramework’). WinFormsObject(’splitContainer1′).WinFormsObject(’SplitterPanel’, ”, 1).WinFormsObject(’Navigation’). WinFormsObject(’CatalogTab’).WinFormsObject(’CatalogFramework’).WinFormsObject(’tableLayoutPanel11′)=WinFormsObject(’FormFramework’).WinFormsObject(’mainContent’).WinFormsObject(’caiTableLayoutPanel1′).WinFOrmsObject(’panel’).WinFormsObject(’CatalogFramework’).WinFormsObject(’tableLayoutPanel11′)

Template Editor|Firewall and QoS|Port Triggering|Table|Service,Table, WinFormsObject(’TemplateEditorFramework’).WinFormsObject(’splitContainer1′).WinFormsObject(’SplitterPanel’, ”, 2). WinFormsObject(’ContentPanel’).WinFormsObject(’PortTriggering’).WinFormsObject(’panel1′).WinFormsObject(’ports’).WinFormsObject(’Port’, ”, %row%+3)

The test script has a similar syntax: the arbitrary descriptive name, an action, and any optional parameters.

Sample Test:

Test=Set Service in Port Trigger to ‘aci’ using drag and drop

Template Editor|Catalogs,Click Template Editor|Catalogs|Services|Tree|TCP,Click
Template Editor|Catalogs|Services|Tree|TCP|,Find,aci
Template Editor|Catalogs|Services|Tree|TCP|%currentEntry%,Click
Template Editor|Catalogs|Services|Tree|TCP|%currentEntry%,Drag, Template Editor|Firewall and QoS|Port Triggering|Table|0|Service

Mats simplifies testing since creating tests does not require programming just simple scripting, the tests are flexible to most application changes, and developers only need modify the script to update, create, and run tests that support changed functionality/implementation.

What can be tested?

• Files
• dBs (via ADO)
• Native Windows applications
• Web applications
• Visual properties of controls (fonts, colors etc.)
• Input/output text (variable text such as user names and fixed text such as prompts for multi-lingual applications)

Here is what the architecture looks like:

Notice that TestComplete is not required to run the tests, the much cheaper TestExecute is all that is needed. At least one instance of TestComplete is required however to record any scripts.

Benefits

There are several advantages that MATS gives to the testing procedure. The use of arbitrary component names permit implementation to be changed without changing tests. Tests are easier to create since they can be written by testers in simple script before the application is available. Testing can easily be performed by developers before committing changes. This assures addition problems are not introduced.

What started as a convenient and cost effective way for an us to offer a value added service has become a product in its own right. We have found customers who are interested in adding this novel approach to automated testing to their QA procedures as a way to cut costs, and expedite testing cycles.

Software programming can be a solitary or collaborative effort. Similarly, so is creating music. The idea for both can be a singular endeavor, but it most often takes a combined effort to reach the finish. This similarity is described in the book by Andrew Johnston and Chris S. Johnson “Extreme Programming: A More Musical Approach to Software Development?“ where they hypothesize on the resemblance between musical and software development environments:

The characterization of software development as a craft rather than an engineering discipline has been proposed recently and it is felt that by considering the similarities and differences between the working and learning environments of musicians and developers that some further weight can be given to calls for a more collaborative approach to software development.

The debate that writing software code is an art has been around almost since coders have existed. I would not go so far as to say anything I have coded is “artistic” but some of the novel ideas that come from very focused coding fit with the idea of creating a piece of music.

Music is the organization of sounds with some degree of rhythm, melody, and harmony. I would argue that harmony should not be included in this definition since even a single voice is considered music where no harmony is present.

But the important idea in this definition is organization: it is the idea of organization and patterns that is common to both music and software. Design patterns are now an integral part of software development. A design pattern is a general reusable solution to a commonly occurring problem in software design. The most famous pattern in music is probably the opening sequence of Beethoven’s Fifth Symphony (G G G E-flat). Beethoven repeats this pattern several times in the movement in several variations. Once we understand the pattern of a particular piece, be it coded or musical, we are well on our way to a solution.

Jonathon Kohl explains the tension/resolution that makes up music is applicable in software testing. I would go further and say that it also applies to software coding. There is the intangible feeling of tension when an algorithm is in development, and the feeling of relief upon its successful resolution. For a great example of tension/resolution in music listen to Lenny Kravitz’ Mr. Cabdriver.

I would like to think that we should not be so pragmatic as to look at software development as being”solving a problem in order to start creating.”

It doesn’t really matter though because when I am done I put on the headphones and crank some Zeppelin. Anybody want to jam?

A problem that often occurs in any software system is related to resource constraints or limitations; how do you make the best use of the resources that you have? This article will examine the challenge of designing a functional web user interface utilizing light server processing.

Modern hardware that handles copious amounts of sensitive data requires faultless processing, uninterruptible operation, and ease of use. The term ‘ease of use’ implies a user interface of some sort, in this case a web GUI. However, a highly interactive web interface can have an impact on the operation of the hardware if general processing is interrupted in order to handle user interface requests. In environments such as HTML: java script(.js), active server pages (.asp), or proprietary GUI development in C++, C# or Visual Basic, how do we maximize user interface functionality while minimizing processing on the server?

Maximize Client Processing

One solution is to limit the amount of processing on the data server that is devoted to the user interface. This may or may not be used with a third party user interface proxy server (UIPS) depending on the environment. A system that expects few concurrent users (typically 5 or less) can operate as:

UI client <–> data server

where there is not a huge demand on the processor.

A system that demands more concurrent users (6 or more) can benefit from a third party UI server operating as:

UI client <–> UIPS <–> data server

that will offload more processing power to the user interface proxy server.

Data Processing in the GUI

Giving some thought as to defining the structure of the data can be very significant in efficient data handling. Web Service Definition Language (WSDL) for SOAP, Management Information Base (MIB) for SNMP or other data syntax design for generic or specific data driven protocols should be given a high priority to assure non-redundant, efficient and light but complete coverage of information. Both the server and the web client will use the same data syntax for their respective communication layers, and as such, a data format that integrates to the system in an efficient way must be chosen.

One of the biggest burdens that is dumped on the server by data acquisition is table processing, especially large two dimensional lists. To alleviate as much overhead as possible make sure any sorting routines are carried out on the client. Table indexing in the data definition can be a big help in facilitating efficient data access.

Scripting languages can be slow and cumbersome. If possible, there should be no scripting engine residing on the server. The data is to be served in as raw a form as possible and all scripting should be carried out by the client.

Statistical data can pose a problem. Multiple server access for several statistics can put a heavy load on the server. Polling intervals should be set for a span that reports relevant information but does not overload the server.

Tradeoffs

These solutions do not come without a price. No server side scripting puts an extra effort on client development. Often the question of adding more data to a network request or supporting functionality on the server must be decided. The GUI itself must handle many of the routines easily handled on the server. This in turn can make the GUI slow and cumbersome. Consequently expert and efficient programming methods will help alleviate this issue.

One of our projects at MapleWorks in the last year required us to integrate the Open Shortest Path First protocol (OSPF) into a residential gateway product. Since OSPF is a mature protocol, and many implementations existed, it was our task to determine which OSPF stack was most suitable given timeline constraints that surround any onshore software development project.

Based on our experience, IP Infusion was the natural choice as it provided everything we needed in a modular architecture. However there was some natural scepticism about our decision, and this resulted in a survey of all the OSPF offerings that we knew of or could find on the Internet. We summarized our findings in the table below. Our survey was performed in May 2007, and much of the accuracy of the information was dependant on technical documentation publicly available at the time.

Description of columns:

OSPF Stack Vendor: IP Infusion, Data Connection, Aricent, XORP, Vyatta, Ospfd, OpenOSPFD, Zebra

Modular Architecture: Is OSPF decoupled from the RTM and other routing protocols? In the future there may be a requirement to have RIP and OSPF running at the same time. A non modular architecture would make it difficult to integrate RIP with OSPF so that both protocols run at the same time.

Management API: Does OSPF come with a clean management API to provision the protocol?

Maturity: How long has the protocol been around? Usually the longer its been around, the more bugs it has resolved making it a more solid product. This is where commercial offerings beat Open Source options.

Cross compiling: Has the code been written to support processors with big endian as well as little endian? Most development is done on PCs which uses little endian, and all too often when code goes from one processor to another, issues with byte alignment arise. These automatically introduce bugs, and will take incur additional costs in time and money.

Open Source: Is the product open source?

Extra cost incurred relative to IP Infusion: Since the IP Infusion stack contained everything that was required for the project, it was used as a baseline to compare against all other offerings.

(Luan Pham is a Senior Software Engineer of MapleWorks Technology Inc, the smart choice for onshore software development for networking and telecom companies.)