Monday, August 10, 2009
How to Test Software when requirements are changing
RAD helps software developers to make the first versions very quickly. This will cause many headaches to testers. With every change there is a possibility to create new defects. The only way to find new defects is to perform a regression test that repeats a series of previous test cases whose results will compare with previous results to find the differences.
In rapid development whether it is possible to test?
Truly say no. This one is a tricky question because even in a stable environment it is not possible to test clean. So in rapid change environment the question could be asked: "Is it possible to test effectively in the rapidly changing?" Can we expect to make the best use of people and other resources to test the software? Can we expect to find the number of defects?
In RAD the process control is essential to find defects with any degree of effectiveness. Since the standard is not to have a repeatable process for most of what we do in building software, many people in a test environment in RAD try a few test cases here and there and look for defects. Which is very difficult to find out the defects?
What strategies can be used?
It takes some time to study which method will works in each which environment. The case is that there should be a unique strategy for a unique system or environment. But there are some strategies that can be used for testing during the rapid evolution:
First, accept the fact that there is no time of luxury to conduct a six-week test for software which changes daily. That means there is a need to define a testing process that can be performed rapidly and efficiently.
Second, perform a risk assessment. Knowing the level of risk is crucial, because the need to prioritize the efforts to adapt to the test in a short time window. Higher the risk, more testing should be done.
Automate your tests. Capture / playback tools help to make the tests repeatable and unattended in a session. Good tools require a significant investment in software and training, but it beats working 24 hours a day. There are things to consider before automation:
There should be a basic version of the software for comparison with future tests.
Requirements, test cases and test scenarios should be defined. The tool can record and playback, depending on what the user performed actions.
The data is a key element. Keeping the data is the main element is testing.
It takes time and lots of money to integrate the tool in your organization. People need to be trained to use the tool. In addition, people need to be sold on long-term benefits in relation to short-term work needed to install test scripts and test.
Conclusion
Testing during the rapid evolution is possible, but it requires a rapid response, smart work, and traceability. Organizations that is not willing to consider new technologies such as automated testing tools will not be able to test effectively during rapid change. It is like building a house with hand tools - which it can be done, eventually.
Testing during the rapid development also requires a new mindset and organizational processes. Tools are not the answer. There must be a process that can be executed quickly and makes the best use of people and time. It is to the optimal mix of tools, processes, and people that make the challenge done.
Friday, August 7, 2009
Some of the Risks associated with software product
Few generic risks related with the size of the product are:
o Estimated size of the product and assurance in estimated size?
o Estimated size of product?
o Size of the database created or used by the product?
o Number of users of the product?
o Number of projected changes to the requirements for the product?
Risk will be high, when a large difference is observed between expected results and the results from the past experience. It is good practice to compare expected information with previous experience for carrying out the analysis of risk.
2) Business Impact Risks:
Few generic risks associated with the business impact are:
o Effect of the software product on income of the company?
o Reasonability for delivery dates?
o Number of clients expected to use the product.
o Stability in the needs of the customer’s comparative to the product?
o Number of other products / systems with which the concerned product is expected to be interoperable?
o Amount and quality of product which must be produced and delivered to the customer?
o Costs associated with overdue delivery or a faulty product?
3) Customer-Related Risks:
Special customers have special needs. Every customer is independent. Some customers readily accept what is delivered to them. Some others complain about the value of the product. In some other cases, customers may have very good relationship with the product and the producer and some other customers may not have. A bad customer represents a major threat to the project plan and a considerable risk for the project manager.
Some checklist for the customer
o Have you engaged with the customer in the past?
o Does the customer have a good idea of his requirement?
o Will the customer agree to spend time for requirement discussions?
o Is the customer enthusiastic to join in reviews?
o Is the customer technically familiar in the product area?
o Does the customer realize the software engineering process?
4) Process Related Risks:
Risks are soaring for software product. if the software engineering method is ill defined or if analysis, design and testing are not conducted in a intended fashion.
o Whether the organization has documented software development process?
o Whether the team members are following the documented software development process?
o Whether the third party programmers are also following the defined software development process.
o Whether keeping a track on the performance of third party programmers?
o Whether the development teams and testing teams are conducting official technical reviews at regular intervals?
o Whether results of every official technical review are properly documented?
o Whether configuration management is used to maintain consistency among system?
o Is there any mechanism for controlling changes to customer necessities which have impact on the software product?
5) Technology Related Risks:
o Whether the technology used is new to the organization?
o Whether the software has proper interface with new hardware configurations?
o Whether the software has proper interface with the database system whose function and performance have not been proven in the concerned application area?
o Whether any specialized user interfaces have been demanded by product requirements?
o Do requirements demand the use of any new analysis, design or testing methods?
o Do requirements put excessive performance constraints on the product?
6) Technical Risks:
o Whether specific methods used for software analysis?
o Whether specific conventions for code documentation defined and used?
o Whether any specific methods used for test case design?
o Whether software tools used to support planning and tracking activities?
o Whether configuration management tools used to control and track change activity throughout the software development process?
o Whether tools used to generate software prototypes?
o Whether tools used to support the testing process?
o Whether tools used to support the production and management of documentation?
o Whether quality metrics collected for all software projects?
o Whether productivity metrics collected for all software projects?
7) Environmental Risks:
o Whether a software project and process management tool available in the organization?
o Whether tools for analysis and design are available in the organization?
o Whether analysis and design tools deliver methods are appropriate for the product to be built?
o Whether compilers or code generators are available for the product to be built?
o Whether testing tools are available for the product to be built?
o Whether software configuration management tools are available in the organization?
o Whether the environment needs a database or repository?
o Whether all software tools are properly incorporated among all?
o Whether all members of the project team have received training on every tool?
8) Team Associated Risks:
o Whether best people are available in enough numbers for the project?
o Do the people have the right mixture of skills?
o Whether all team members are committed for the entire duration of the project?
Thursday, August 6, 2009
Steps involved in Testing
esting principally consists of two key activities they are 1) Organising Sandboxes 2) Developing Test Cases.
1): Organizing Sandboxes: Database testing involves the need of a copy of databases which are called sandboxes.
These sandboxes are of following three types
a) Functionality Sandbox: In this the new functionality of database will be checked and reflected in the existing functionality. Then the tested sandbox will pass to the next stage, which is an integrated sandbox.
b) Integrated Sandbox: In this stage all the sandboxes will be got integrated and then test the system.
c) QA sandbox: After the system is tested, sandboxes are sent for acceptance testing. This will ensure the quality of the database.
2) Development of test cases: The step by step process for the development of test cases is as under:
The first step is setting up of the test cases: Set up the database to a known state.
The sources of test data are
1) External test data.
2) Test scripts.
3) Test data with known values.
4) Real world data.
The second step is running the test cases: The test cases are then run. The running of the database test cases is analogous to usual development testing.
Traditional Approach of Test Case Execution:
Test cases are executed on the client side. The results which produced are then validated against expected values.
Advantages of Traditional Approach: It is simple and no programming skill is required. It not only addresses the functionality of stored procedures, rules, triggers and data integrity but also the functionality of application as a whole.
Disadvantages of Traditional Approach:
1) Sometimes the results after test case execution do no necessarily indicate that the data itself is properly written to a record in the table.
2) When wrong results are sent back after the execution of test cases, it doesn't necessarily mean that the error is a database error.
3) A crucial danger with database testing and with regression testing in specific is coupling between tests. If we put the database in to a known state, run several tests against that known states, before setting it, then those tests are potentially coupled to one another.
Advanced Approach of Test Case Execution:
First of all we need to do a schematic preparation for Testing, which involves:
Generate a list of stored procedures, triggers, defaults, rules and so on. This will help us to have a good handle on the scope of testing required for database testing.
Thereafter we can follow the following points:
1. Generate a scheme for test case table. Analyzing the scheme will help us determine the following:
- Can any field be marked as NULL?
- Boundary values.
- Constraints.
- Connectivity of various variables?
- Is there any look Up table available to check?
- What are user defined data types?
- What are primary key and foreign key relationships among tables?
- What is the primary function of each function? Does it read data and produce outputs, write data or both?
- What are the accepted parameters?
- What are the return values?
- When is the stored procedure called and by whom?
- When is a trigger fired?
Third step is to Check the results: Actual database test results and expected database test results are compared.
Wednesday, August 5, 2009
Statistical control
Traditional statistical process controls in manufacturing operations usually proceed by randomly sampling and testing a fraction of the output. Variances of critical tolerances are continuously tracked, and manufacturing processes are corrected before bad parts can be produced.
Statistical Process Control (SPC) is an effective method of monitoring a method through the use of organizes charts. Organize charts allow the use of objective criteria for distinguishing background distinction from actions of impact based on statistical techniques. Much of its control lies in the capability to monitor both process center and its variation about that center. By collecting data from samples at various points within the process, variations in the process that may affect the quality of the end product or service can be detected and corrected, thus reducing waste as well as the likelihood that problems will be passed on to the customer. With its emphasis on early detection and prevention of problems, SPC has a distinct advantage over quality methods, such as inspection, that apply resources to detecting and correcting problems in the end product or service.
In addition to reducing waste, SPC can lead to a reduction in the time required to produce the product or service from end to end. This is partially due to a diminished likelihood that the final product will have to be reworked, but it may also result from using SPC data to identify bottlenecks, wait times, and other sources of delays within the process. Process cycle time reductions coupled with improvements in yield have made SPC a precious tool from both a cost reduction and a customer satisfaction standpoint.
History
Statistical Process Control was pioneered by Walter A. Shewhart in the early 1920s. The concept of quality control in manufacturing was first advanced by Walter Shewhart The first to apply the newly discovered statistical methods to the problem of quality control was Walter A. Shewhart of the Bell Telephone Laboratories. He issued a memorandum on May 16, 1924 that featured a sketch of a modern control chart. W. Edwards Deming later introduced SPC methods in the United States during World War II, thereby successfully improving quality in the manufacture of weapons and other strategically important products. Deming was also instrumental in introducing SPC methods to Japanese industry after the war had ended.
Shewhart formed the basis for the control chart and the concept of a state of statistical control by carefully designed experiments. While Dr. Shewhart drew from pure mathematical statistical theories, he understood that data from physical processes seldom produces a "normal distribution curve". He revealed that observed variation in manufacturing data did not always behave the same way as data in nature. Dr. Shewhart concluded that while every process displays variation, some processes display controlled variation that is natural to the process, while others display uncontrolled variation that is not present in the process causal system at all times .
In 1989, the Software Engineering Institute introduced the idea that SPC can be applied to non-manufacturing processes, such as software engineering processes, in the Capability Maturity Model (CMM). This idea exists today within the Level 4 and Level 5 practices of the Capability Maturity Model Integrated (CMMI).
General
In mass-manufacturing, the worth of the completed article was usually achieved through post-manufacturing inspection of the product; accepting or rejecting each piece based on how well it met its design specifications. In contrast, Statistical Process Control uses statistical tools to observe the performance of the production process in order to predict significant deviations that may later result in rejected product.
Two kinds of variation occur in all manufacturing processes. The first is known as natural or common cause variation and may be variation in temperature, difference in raw materials, voltage. This variation is minute, the observed values generally being quite close to the average value. The second kind of variation is known as special cause variation, and happens less frequently than the first.
How to Use SPC
Initially, one starts with a quantity of data from a manufacturing process with a definite metric, i.e. mass, length, surface energy of a widget. There should be upper threshold and lower threshold. The Upper threshold Limits of the process would be set to average plus three units and the Lower Control Limit would be set to average minus three units. The action taken depends on gauge and where each run lands on the SPC chart in order to control but not tamper with the process.
After some times other process-monitoring tools have been developed, including:
Cumulative Sum (CUSUM) charts: the ordinate of each plotted point represents the algebraic sum of the previous ordinate and the most recent deviations from the target.
Exponentially Weighted Moving Average (EWMA) charts: each chart point represents the weighted average of current and all previous subgroup values, giving more weight to recent process history and decreasing weights for older data.
Failure testing
Failure testing is an important part of the manufacturing process, no matter what are manufacturing. Failure testing is a way to ensure that the product and service that will not fail under different circumstances and situations of stress, weather, temperature, and so on and so forth. Continuous failure testing, even after a product is developed, will help you ensure that your manufacturing processes are on optimum possible way and that which continually improving the products and the services.
When a product fails, then examine those failures quickly so that the problem corrected. When failure testing is performing on a component that has failed, or just to test for a known failure, then it is very necessary to correlate the observations of a number of unusual aspects of the module: the appearance of the component or product, its composition, and its strength. Also keep in mind the design of the product, the operating conditions, the service environment, and the manufacturing record.
Failure testing measures enclose many of the same components and practices as failure examination. Failure analysis occurs after the fact, but failure testing strives to occur before the fact so that failure can hopefully be avoided by continually testing products and components so that they can be improved before they fail. It would be beneficial to you and your customers if you engage in failure testing on a regular basis, so that you can prevent any future problems. There are a number of different ways that you can go through failure testing; the best approaches to failure testing will be specific to your particular industry. A good overall approach to manufacturing processes, such as lean manufacturing or Six Sigma, will include failure testing as a part of its approach to industrialized process management.
Ensure the test to be conduct so that, to take into consideration material composition, the macro structure and the micro structure of the particular component, the distribution of hardness, the mechanical properties of the component, how well the element or the product resists deterioration, what happens when the product is put into prolonged contact with saline, the consequence of moisture on the product or the component, different environmental exposures, and what happens when the product is confronted with abrasives. It is advised to look carefully at fatigue, fracture testing, the flexural, yield, and ultimate strength of your product or component, the impact strength and corrosion resistance of every component of your product, and more.
Failure testing will help to ensure the quality of the products and the services. Regular failure testing will be a preventative measure than a necessarily corrective measure that occurs after the potentially disastrous fact. Quality within a business is usually defined in terms of the relation between the customer, the process or product, and the business.
Stress is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. Stress testing a subset of load testing
Stress testing is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. Stress testing may have a more specific meaning in certain industries.
I'm sure devious testers can enhance this list with their favorite ways of breaking systems.
Stress testing practices employed prior to the start of the crisis in four broad areas: (i) use of stress testing and integration in risk governance; (ii) stress testing methodologies; (iii) scenario selection; and (iv) stress testing of specific risks and products.
Tuesday, August 4, 2009
Quality assurance activitiy
One of the commonly used prototypes for QA management is the PDCA (Plan-Do-Check-Act) approach.
Plan–Do–Check–Act Cycle
The concept of the PDCA Cycle was originally developed by Walter Shewhart, the pioneering statistician who developed statistical process control in the Bell Laboratories in the
Description
The plan–do–check–act cycle (Figure 1) is a four-step model for carrying out change. It is just as a circle that has no end, the PDCA cycle should be repeated again and again for continuous improvement. Use the PDCA Cycle to coordinate your continuous improvement efforts. It both emphasizes and demonstrates that improvement programs must start with careful planning, must result in effective action, and must move on again to careful planning in a continuous cycle.
Figure 1: Plan-do-check-act cycle
When Plan-Do-Check-Act come into play?
It is a model of continuous improvement.
When the starting of a new improvement project.
When developing a new or improved design of a process, product or service.
When defining a repetitive work process.
When planning data collection and analysis in order to verify and prioritize problems or root causes.
When implementing any change.
Plan-Do-Check-Act Procedure
Plan. Recognize an opportunity and plan a change. Plan to improve your operations first by finding out what things are going wrong (that is identify the problems faced), and come up with ideas for solving these problems.
Do. Test the change. Carry out a small-scale study. Do changes designed to solve the problems on a small or experimental scale first. This minimises disruption to routine activity while testing whether the changes will work or not.
Study. Review the test, analyze the results and identify what you’ve learned. Check whether the small scale or experimental changes are achieving the desired result or not. Also, continuously Check nominated key activities (regardless of any experimentation going on) to ensure that you know what the quality of the output is at all times to identify any new problems when they crop up.
Act. Take action based on what you learned in the study step: If the change did not work, go through the cycle again with a different plan. Act to implement changes on a larger scale if the experiment is successful. This means making the changes a routine part of your activity.
. The diagram below lists the tools and techniques which can be used to complete each stage of the PDCA Cycle.
Quality Management Components
Quality Management Components
Quality control is a process engaged to make sure a certain level of quality in a product or service. It consists whatever actions a industry deems crucial to provide for the control and verification of certain characteristics of a product or service. The basic goal of quality control is to make sure that the products provided meet specific requirements.
Quality control involves the examination of a product for certain minimum levels of quality. The goal of a quality control team is to identify products that do not meet a company's specified standards of quality. If a problem is recognized, the job of a quality control team is like that the quality to prescribe to stop production temporarily. It depends on the particular product, type of problem identified.
Generally, it is not the job of a quality control team to correct quality issues. Other individuals, who are involved, discover the cause of quality issues and fix them. Once these kinds of problems are overcome, the product continues production or implementation as planned.
Quality control can cover not only products, services, and processes, but also people. Employees are an important part of the company. If a company has employees that don't have adequate skills or training, have trouble understanding directions, or are misinformed, quality may be severely diminished. When quality control is considered about human beings, it concerns correctable issues.
Often, quality control is confused with quality assurance. Even if the two are very similar, there are some basic differences. Quality control is concerned with the product and quality assurance is process oriented.
Even with such a clear-cut difference defined, identifying the differences among the two is hard. Basically, quality control involves evaluating a product. Quality assurance is designed to make sure processes are sufficient to meet objectives. Quality assurance ensures a product is manufactured, implemented, created, or produced in the right way. Quality control evaluates whether the end result is satisfactory.
Quality assurance (QA) is the activity of providing evidence needed to establish quality in work. The activities that require good quality are being performed effectively. All the systematic actions, which provide enough confidence, that a product will satisfy the given requirements for quality.

