Abstract: In the current technological market environment, ensuring the quality of new products has become a complex challenge. In this scenario, companies have been investing in solutions that aim to reduce the execution time of software testing and lead to cost efficiency. However, companies that have a complex and specialized testing environment usually face barriers related to costly testing processes, especially in distributed settings. Sidia Institute of Technology works on research and development for the Android platform for mobile devices in Latin America. As we work in a global software development (GSD) scope, we have faced barriers caused by failures detected lately that have caused delays in the homologation release process on Android projects. Thus, we adopt an Internal Review process, using as an alternative to reduce these failures. In this paper it was presented the experience of a homologation team adopting an Internal Review process in order to increase the performance through of improving test efficiency. Using this approach, it was possible to realize a substantial improvement in quality, reliability and timeliness of our deliveries. Through the quantitative analyses, it was possible identify a positive growth in homologation efficiency of 6% after adoption of the process. In addition, we performed a qualitative analysis from the collected data through an online questionnaire. In particular, results show that association between failure reduction and review process adoption provides the most quality that has a positive effect on project milestones. We hope this report can be helpful to other companies and the scientific community to improve their process thereby increasing competitive advantages.
Abstract: Software bug localization is one of the most costly tasks in program repair technique. Therefore, there is a high claim for automated bug localization techniques that can monitor programmers to the locations of bugs, with slight human arbitration. Spectrum-based bug localization aims to help software developers to discover bugs rapidly by investigating abstractions of the program traces to make a ranking list of most possible buggy modules. Using the Apache Commons Math library project, we study the diagnostic accuracy using our spectrum-based bug localization metric. Our outcomes show that the greater performance of a specific similarity coefficient, used to inspect the program spectra, is mostly effective on localizing of single line bugs.
Abstract: This paper is aimed at creating an Automatic Java X-Machine testing tool for software development. The nature of software development is changing; thus, the type of software testing tools required is also changing. Software is growing increasingly complex and, in part due to commercial impetus for faster software releases with new features and value, increasingly in danger of containing faults. These faults can incur huge cost for software development organisations and users; Cambridge Judge Business School’s research estimated the cost of software bugs to the global economy is $312 billion. Beyond the cost, faster software development methodologies and increasing expectations on developers to become testers is driving demand for faster, automated, and effective tools to prevent potential faults as early as possible in the software development lifecycle. Using X-Machine theory, this paper will explore a new tool to address software complexity, changing expectations on developers, faster development pressures and methodologies, with a view to reducing the huge cost of fixing software bugs.
Abstract: Growing dependency of mankind on software
technology increases the need for thorough testing of the software
applications and automated testing techniques that support testing
activities. We have outlined our testing strategy for performing
various types of automated testing of Java applications using
AspectJ which has become the de-facto standard for Aspect Oriented
Programming (AOP). Likewise JUnit, a unit testing framework is
the most popular Java testing tool. In this paper, we have evaluated
our proposed AOP approach for automated testing and JUnit on
various parameters. First we have provided the similarity between
the two approaches and then we have done a detailed comparison
of the two testing techniques on factors like lines of testing code,
learning curve, testing of private members etc. We established that
our AOP testing approach using AspectJ has got several advantages
and is thus particularly more effective than JUnit.
Abstract: Manual writing of test cases from functional requirements is a time-consuming task. Such test cases are not only difficult to write but are also challenging to maintain. Test cases can be drawn from the functional requirements that are expressed in natural language. However, manual test case generation is inefficient and subject to errors. In this paper, we have presented a systematic procedure that could automatically derive test cases from user stories. The user stories are specified in a restricted natural language using a well-defined template. We have also presented a detailed methodology for writing our test ready user stories. Our tool “Test-o-Matic” automatically generates the test cases by processing the restricted user stories. The generated test cases are executed by using open source Selenium IDE. We evaluate our approach on a case study, which is an open source web based application. Effectiveness of our approach is evaluated by seeding faults in the open source case study using known mutation operators. Results show that the test case generation from restricted user stories is a viable approach for automated testing of web applications.
Abstract: Test automation allows performing difficult and time
consuming manual software testing tasks efficiently, quickly and
repeatedly. However, development and maintenance of automated
tests is expensive, so it needs a proper prioritization what to automate
first. This paper describes a simple yet efficient approach for such
prioritization of test cases based on the effort needed for both manual
execution and software test automation. The suggested approach is
very flexible because it allows working with a variety of assessment
methods, and adding or removing new candidates at any time. The
theoretical ideas presented in this article have been successfully
applied in real world situations in several software companies by the
authors and their colleagues including testing of real estate websites,
cryptographic and authentication solutions, OSGi-based middleware
framework that has been applied in various systems for smart homes,
connected cars, production plants, sensors, home appliances, car head
units and engine control units (ECU), vending machines, medical
devices, industry equipment and other devices that either contain or
are connected to an embedded service gateway.
Abstract: Software testing has become a mandatory process in
assuring the software product quality. Hence, test management is
needed in order to manage the test activities conducted in the
software test life cycle. This paper discusses on the challenges faced
in the software test life cycle, and how the test processes and test
activities, mainly on test cases creation, test execution, and test
reporting is being managed and automated using several test
automation tools, i.e. Jira, Robot Framework, and Jenkins.
Abstract: Assertion-Based software testing has been shown to
be a promising tool for generating test cases that reveal program
faults. Because the number of assertions may be very large for
industry-size programs, one of the main concerns to the applicability
of assertion-based testing is the amount of search time required to
explore a large number of assertions. This paper presents a new
approach for assertions exploration during the process of Assertion-
Based software testing. Our initial exterminations with the proposed
approach show that the performance of Assertion-Based testing may
be improved, therefore, making this approach more efficient when
applied on programs with large number of assertions.
Abstract: Since Software testing becomes an important part of
Software development in order to improve the quality of software,
many automation tools are created to help testing functionality of
software. There are a few issues about usability of these tools, one is
that the result log which is generated from tools contains useless
information that the tester cannot use result log to communicate
efficiently, or the result log needs to use a specific application to open.
This paper introduces a new method, SBTAR that improves usability
of automated test tools in a part of a result log. The practice will use
the capability of tools named as IBM Rational Robot to create a
customized function, the function would generate new format of a
result log which contains useful information faster and easier to
understand than using the original result log which was generated
from the tools. This result log also increases flexibility by Microsoft
Word or WordPad to make them readable.
Abstract: Enterprise Architecture (EA) is a framework for description, coordination and alignment of all activities across the organization in order to achieve strategic goals using ICT enablers. A number of EA-compatible frameworks have been developed. We, in this paper, mainly focus on Federal Enterprise Architecture Framework (FEAF) since its reference models are plentiful. Among these models we are interested here in its business reference model (BRM). The test process is one important subject of an EA project which is to somewhat overlooked. This lack of attention may cause drawbacks or even failure of an enterprise architecture project. To address this issue we intend to use International Software Testing Qualification Board (ISTQB) framework and standard test suites to present a method to improve EA testing process. The main challenge is how to communicate between the concepts of EA and ISTQB. In this paper, we propose a method for integrating these concepts.
Abstract: An application framework provides a reusable
design and implementation for a family of software systems.
Frameworks are introduced to reduce the cost of a product line
(i.e., family of products that share the common features). Software
testing is a time consuming and costly ongoing activity during the
application software development process. Generating reusable test
cases for the framework applications at the framework
development stage, and providing and using the test cases to test
part of the framework application whenever the framework is used
reduces the application development time and cost considerably.
Framework Interface Classes (FICs) are classes introduced by
the framework hooks to be implemented at the application
development stage. They can have reusable test cases generated at
the framework development stage and provided with the
framework to test the implementations of the FICs at the
application development stage. In this paper, we conduct a case
study using thirteen applications developed using three
frameworks; one domain oriented and two application oriented.
The results show that, in general, the percentage of the number of
FICs in the applications developed using domain frameworks is, on
average, greater than the percentage of the number of FICs in the
applications developed using application frameworks.
Consequently, the reduction of the application unit testing time
using the reusable test cases generated for domain frameworks is,
in general, greater than the reduction of the application unit testing
time using the reusable test cases generated for application
frameworks.
Abstract: This paper presents the design and implementation of CASTE, a Cloud-based automatic software test environment. We first present the architecture of CASTE, then the main packages and classes of it are described in detail. CASTE is built upon a private Infrastructure as a Service platform. Through concentrated resource management of virtualized testing environment and automatic execution control of test scripts, we get a better solution to the testing resource utilization and test automation problem. Experiments on CASTE give very appealing results.
Abstract: Testing is an activity that is required both in the
development and maintenance of the software development life cycle
in which Integration Testing is an important activity. Integration
testing is based on the specification and functionality of the software
and thus could be called black-box testing technique. The purpose of
integration testing is testing integration between software
components. In function or system testing, the concern is with overall
behavior and whether the software meets its functional specifications
or performance characteristics or how well the software and
hardware work together. This explains the importance and necessity
of IT for which the emphasis is on interactions between modules and
their interfaces. Software errors should be discovered early during
IT to reduce the costs of correction. This paper introduces a new type
of integration error, presenting an overview of Integration Testing
techniques with comparison of each technique and also identifying
which technique detects what type of error.
Abstract: Testable software has two inherent properties – observability and controllability. Observability facilitates observation of internal behavior of software to required degree of detail. Controllability allows creation of difficult-to-achieve states prior to execution of various tests. In this paper, we describe COTT, a Controllability and Observability Testing Tool, to create testable object-oriented software. COTT provides a framework that helps the user to instrument object-oriented software to build the required controllability and observability. During testing, the tool facilitates creation of difficult-to-achieve states required for testing of difficultto- test conditions and observation of internal details of execution at unit, integration and system levels. The execution observations are logged in a test log file, which are used for post analysis and to generate test coverage reports.
Abstract: This paper proposes a new technique for improving
the efficiency of software testing, which is based on a conventional
attempt to reduce test cases that have to be tested for any given
software. The approach utilizes the advantage of Regression Testing
where fewer test cases would lessen time consumption of the testing
as a whole. The technique also offers a means to perform test case
generation automatically. Compared to one of the techniques in the
literature where the tester has no option but to perform the test case
generation manually, the proposed technique provides a better
option. As for the test cases reduction, the technique uses simple
algebraic conditions to assign fixed values to variables (Maximum,
minimum and constant variables). By doing this, the variables values
would be limited within a definite range, resulting in fewer numbers
of possible test cases to process. The technique can also be used in
program loops and arrays.
Abstract: An application framework provides a reusable design
and implementation for a family of software systems. Frameworks
are introduced to reduce the cost of a product line (i.e., a family of
products that shares the common features). Software testing is a timeconsuming
and costly ongoing activity during the application
software development process. Generating reusable test cases for the
framework applications during the framework development stage,
and providing and using the test cases to test part of the framework
application whenever the framework is used reduces the application
development time and cost considerably. This paper introduces the
Framework Interface State Transition Tester (FIST2), a tool for
automated unit testing of Java framework applications. During the
framework development stage, given the formal descriptions of the
framework hooks, the specifications of the methods of the
framework-s extensible classes, and the illegal behavior description
of the Framework Interface Classes (FICs), FIST2 generates unitlevel
test cases for the classes. At the framework application
development stage, given the customized method specifications of
the implemented FICs, FIST2 automates the use, execution, and
evaluation of the already generated test cases to test the implemented
FICs. The paper illustrates the use of the FIST2 tool for testing
several applications that use the SalesPoint framework.
Abstract: Software reliability, defined as the probability of a
software system or application functioning without failure or errors
over a defined period of time, has been an important area of research
for over three decades. Several research efforts aimed at developing
models to improve reliability are currently underway. One of the
most popular approaches to software reliability adopted by some of
these research efforts involves the use of operational profiles to
predict how software applications will be used. Operational profiles
are a quantification of usage patterns for a software application. The
research presented in this paper investigates an innovative multiagent
framework for automatic creation and management of
operational profiles for generic distributed systems after their release
into the market. The architecture of the proposed Operational Profile
MAS (Multi-Agent System) is presented along with detailed
descriptions of the various models arrived at following the analysis
and design phases of the proposed system. The operational profile in
this paper is extended to comprise seven different profiles. Further,
the criticality of operations is defined using a new composed metrics
in order to organize the testing process as well as to decrease the time
and cost involved in this process. A prototype implementation of the
proposed MAS is included as proof-of-concept and the framework is
considered as a step towards making distributed systems intelligent
and self-managing.
Abstract: The users are now expecting higher level of
DSP(Digital Signal Processing) software quality than ever before.
Prevention and detection of defect are critical elements of software
quality assurance. In this paper, principles and rules for prevention and
detection of defect are suggested, which are not universal guidelines,
but are useful for both novice and experienced DSP software
developers.
Abstract: In spite of all advancement in software testing,
debugging remains a labor-intensive, manual, time consuming, and
error prone process. A candidate solution to enhance debugging
process is to fuse it with testing process. To achieve this integration,
a possible solution may be categorizing common software tests and
errors followed by the effort on fixing the errors through general
solutions for each test/error pair. Our approach to address this issue is
based on Christopher Alexander-s pattern and pattern language
concepts. The patterns in this language are grouped into three major
sections and connect the three concepts of test, error, and debug.
These patterns and their hierarchical relationship shape a pattern
language that introduces a solution to solve software errors in a
known testing context.
Finally, we will introduce our developed framework ADE as a
sample implementation to support a pattern of proposed language,
which aims to automate the whole process of evolving software
design via evolutionary methods.
Abstract: State-based testing is frequently used in software testing. Test data generation is one of the key issues in software testing. A properly generated test suite may not only locate the errors in a software system, but also help in reducing the high cost associated with software testing. It is often desired that test data in the form of test sequences within a test suite can be automatically generated to achieve required test coverage. This paper proposes an Ant Colony Optimization approach to test data generation for the state-based software testing.