Continuous testing: A paradigm change in testing

We have been working with global customers across regions setting up outcome based test center of excellence. Today agile development methodology is being used or will be used by most of the customers. Independent testing teams have to work in an embedded and integrated manner within scrum teams. Also organizations are moving towards continuous delivery. This requires a different approach towards testing. Off late i am spending lot of time with my team on developing a test framework that can be used for continuous delivery for a test organization which will result in an independent, integrated and embedded test entity. We are calling the testing approach as continuous testing. This is a mixture of both manual functional testing and test automation at different levels. To enable continuous delivery, testing should be done at different levels with most of it being automated.

A paradigm shift should be brought to enable continuous delivery in an organization. This requires a mindset change across the IT organization and view testing as an activity that enhances the overall productivity and quality of IT delivery. Testing should be looked as a strategic activity to ensure that the right investments are made. Few challenges that we identified to enable continuous testing on which we invested are listed here

1.  The knowledge gap of the tester: In agile development, stress is more put on creating documentation that would drive a discussion. This would mean that testers cannot expect a detailed requirement specification document or functional design document which they can use to prepare test cases. Also the turnaround time for tester is limited and he needs to turn around test cases, test results and automation all within the sprint. To make this happen we had to ensure that this knowledge gap that is normally observed in test teams is overcome

2.       Who owns the quality of systems in production? : This has been hotly debated. I might get into trouble for bringing this up. In an organization, this question should be answered and the response should be unambiguous. There might be multiple teams or functions responsible for quality or contributing towards quality but accountability should rest with just one team. That team should take the ownership and drive other teams towards improvement. This has been our personal experience and fixing this will solve many problems. This team should be empowered to take the decisions that will improve quality. This one decision will take the organization into virtuous orbit resulting in a domino effect across all other functions

3.       The perpetual automation backlog: Test Automation is indisputably our biggest challenge. Traditionally automation has been an afterthought. Once an end to end business process is ready it would be flipped over to the automation team who will start with understanding the business process. Also lot of stress is put on identifying the business case for automation. Once there is a business case approved, test automation is taken up as a separate project. Later once automation is completed it is added as part of the regression suite and executed there after forever with no updates (in most cases). But for continuous delivery using agile development, test automation cannot be a one off project that is done on a cost benefit analysis decision.Automation has always been lagging when compared with development or manual testing. For success with continuous delivery this gap should be minimized or completely eliminated. At one level, it does need change in the way we look at automation and the framework but the biggest challenge that we had to overcome was how do we does this within the sprint. How do we enable our teams at sprint level to automate the user stories across multiple teams under the constraints? We are in the process of changing the mindset of both functional tester (FT) and automation tester (AT) and motivating them to become ‘FAT’ testers (functional automation testers)

4.       Flexible automation framework: Automation frameworks are traditionally built to manage end to end business processes that run during regression testing. The supporting automation for test automation like test data automation, exception handling, start stop scripts will change when we look at automating user stories and use that as the basis for building end to end automation for cross functional business processes. It gets further complicated when the automation scripts needs to cut across multiple package applications and also automation scripts written in different technologies need to be integrated to form an end to end chain. This framework is called as the ‘CONTest framework’ (continuous testing framework)

5.       Develop a balanced view on nonfunctional testing: Nonfunctional testing is becoming extremely important in the digital era. But nonfunctional testing can really the exchequer if we go overboard on it mainly due to the tools used in this area. Most of the open source tools are not very effective and licensed tools are exorbitantly expensive and the volumes do not justify the investment. So it is important to do a profiling of the application for various parameters like performance, security, load etc and determine the usage patterns. This should be done at the starting of the project and this requires a mindset change in the development organization. So change for the test team to establish the right nonfunctional requirements and test them at regular intervals

6.       Look beyond functional quality: All said and done testing as an execution activity comes only after development. Our challenge was to come up with ideas that will intercept the development activities much earlier. But at the same time over doing this activity will increase the cost tremendously. So essentially we had to overcome 3 challenges. First challenge was to identify areas where we as independent testing team can intercept before we start test execution and provide improvements which are preventive in nature. Second challenged was to find optimal balance between velocity and cost to bring in these preventive activities. Third challenge was combine these findings with functional findings that we would find during our regular functional testing.

7.       Integrated test tooling: To enable continuous delivery, clarity on end to end process value chain is required. Once this end to end value chain is clear, the process should be automated using tools. Customers traditionally have invested in point tools which do a certain activity well. The tools usage also has led to silo teams. BAs use a requirements management or modeling tool, developers use a IDE and configuration management tool and testers use a test management and test automation tool. All these tools are standalone and don’t talk to each other. So it is important and also a challenge to create this end to end to value chain and tag the change to test so it assure business about quality.

These are challenges that large organizations face at enterprise level. Continuous testing cannot be just achieved by test automation. Other areas like environment provisioning, deployment, build management should be automated to take benefit from the high levels of test automation. May be these are topics for future blogs.

Any critique and comments are sincerely appreciated.

PS: We were able to overcome most of the challenges mentioned above and are moving forward to reach next levels of maturity which is enabling us to help the customer implement a virtuous improvement cycle across the enterprise.

Are we nurturing the roots?

Thanks for reading my last blog and providing feedback. Appreciate your time spent doing the same. I should apologize that i did not mention my colleague Venkat's name in my last blog. I sincerely thank him for improving my English and sent formation. It made it more readable.

In this post i want to focus on something which so important for organizations and teams. This is about nurturing the roots of any organization. 

Plant's root is that part, which in most cases is below the soil and not seen. It is tucked away below the earth reaching to great depths doing its job silently unnoticed. Roots have four major functions. 

1) absorption of water and inorganic nutrients
2) anchoring of the plant body to the ground, and supporting it
3) storage of food and nutrients 
4) vegetative reproduction

Why am i writing about botany you might be wondering. No i am not. I hated it in school and college even though my father was a agriculturist. The beauty about roots is that, it does not matter how big the tree might be, but still you water the roots. You don't water the leaves or any other part of the plant. The strength of the plant is determined by the health of its roots. Getting to the point, this so true with organizations. In an organization context, its roots are its PEOPLE. Are they being nurtured well? They are away from the front page of news papers or television screens. As in the case of roots, they are hidden and provide the required nourishment to the organization. I focus on two aspects. It is important that organization and the leaders spend time to nurture roots. 

Let me first take one aspect, which the organizations can do to bring the required focus in providing the employee 'nourishment'. When i looked at the balance sheet of several companies, i realized that training is not even a line item reported as part of quarterly reports. May be it gets subsumed into other expenses. I feel companies which are serious at ensuring that they have the right focus on empowering employees with required training and knowledge, need to ensure that they have committed investments on training. The shareholders should put pressure on management to report this number and provide enough details, so that all can know how the company is placed to achieve the vision that it has put forth. Shareholders can link the vision with right investments that company is making in training and re-skilling.They can easily evaluate whether the company does 'Walk the Talk'. Ultimately shareholders should be concerned about the sustenance of their own wealth. 

Also companies should be outcome focused when looking at training. Most of the companies look at the number of hours spent on training as a metric. But they should focus on the outcome of the training. 
1. how many employees are using the training in day today activities
2. how many can act as trainers
3. how many can create IP around the new skills by mixing it with their current skills are few options

Moving on to how managers and leaders can improve the nourishment. Managers or leaders should spend enough time in nurturing the right talent. They should spend time in identifying the right candidates with whom they need to spend at least 30% of their productive time. They should spend this time in providing them guidance across different aspects. It can be
1. Helping him to set the right career path and identify the gaps that employees need to work on to take the proposed career path
2. Spend enough time with employees in reviewing the deliverable. This might seem trivial. But i have observed that this is one of the most important activities that a manager should master. This helps him to create the very much needed bandwidth which he can focus to think head. This is also a way to share the expectations from the employee. Through this intervention, the manager or leader can set the bar high by bringing his own learning and visibility.
3. I also believe that managers and leaders should take calculated risk by putting employees into very challenging situations with good intentions. Manager should also provide the employee the required confidence, that in case he is not able to deliver, he would back him up. This commitment is very important to ensure that the employees take up the challenge with confidence.
4. Recognizing employees who achieve the set goals is very important. I feel many managers don't think about this seriously. They are very casual in appreciating the achievements. I feel this does not motivate others to go beyond the call of duty and also to take ownership on outcomes.

To conclude, i believe management and managers should nurture the roots in different ways. The time they spend on nurturing the employees, provides both management and managers immense bandwidth which they can focus on creating sustained competitive advantage.

Agile Series: Do companies note the skill change required for Agile development ? (post 1)

I have been thinking of writing on my Agile experience for the past couple of years. Of-course i have not been Agile on this. Fortunately now after the festival of Sankranti (http://en.wikipedia.org/wiki/Makar_Sankranti) which is celebrated in India and marks the beginning of Sun's northward journey, i have finally made up my mind  to start blogging regularly. I will articulate my reflections which is an outcome of trying to build the agile consulting practice for my employer.

My first tryst with agile software development was in 2007. It seems long time back. The consulting project then was to help the enterprise project management office of a large oil major to develop an approach to adopt agile development. They had a phase gate approach. The typical challenges that was in front of me were

• Develop an approach that captures the best practices of the gated model
• Minimize the challenges of organization change management
• The impact of agile on existing outsourcing contracts

In this blog i want to articulate my view on impact of agile on outsourcing contracts which are biased towards offshoring. Such cost centered contracts drive a certain behavior which does not facilitate agile development. Discussing all areas of impact that Agile development calls for is beyond the scope of this post. Here i want to focus on one important aspect which is not given the required importance. It is about 'People'. I want to take the liberty to quote from A.G Lafley's and R.L Martin book "Playing to Win: How Strategy Really Works" . They make a point that "Leaders at all levels of the company need to be capable strategists and capable operators". When i read this statement it just clicked for me. This is very much applicable to organizations and teams trying to implement agile. 

In Agile teams we need all stakeholders and mainly developers and testers to develop an end to end feel or intuition for the product being developed. This will enable them to think along with the product owner to develop a product road map with technical inputs. Product owner will not have to spend too much time in explaining the intent of requirements or product. Team will be able to make decisions on their own and explain the benefit or impact to product owner. This removes the bottleneck that gets created due to non availability of product owners. This understanding will also help developers and testers to work with support functions like release management or operations to develop release plans. Release plans developed with this learning will truly be logical increments of the product. Such logical increments will also facilitate easy use of techniques like continuous integration and continuous testing. 

This sought of ability to think both strategically and translate that into operational sprints and release plans needs considerable improvement and scaling of people skills. Are customers and outsourcing vendors looking into this seriously. Do the current hiring techniques, training techniques, appraisal techniques support this change? Is there enough focus on developing strategic capability of people at all levels? What is the impact of this on operating margins and rates? So, there is a need to shift from a cost centered outsourcing towards a Value realization mode.

To conclude, it is important for all stakeholders to understand and put people at the center of the contracting discussions. This is not only good for people but also good for balance sheet and for sustained competitive advantage.