Security Testing

Security Testing on Mobile Application

2010-05-06T16:41:00.000+05:30

Security risks associated with mobile applications can often be identified and mitigated by subjecting them security testing. Compared to desktop or web applications mobile applications are harder to test for security and hence a bit neglected.

Application Footprint Analysis
For applications to be installed in mobiles, application footprint analysis begins much before the installation itself. Developers often assume that the phone memory is a safe location and often use it to store user id and passwords and other sensitive information.

When analyzing the phones file system, the main goals are:
1)Identify the files created on the phone by the application during installation. If option available try and install the application in external storage device like flash card etc.. Once this is achieved further analysis like reversing the application, modifying the application and extracting hidden secrets can be performed.

2)Identify changes made to existing filed over multiple application operation.

3)Analyze the info written on the phone file system during various stages of operation.

Perform the following steps to analyze the phones files system:
Step-1 Directory and File Selection: The purpose of directory selection is to reduce the number of phone file system directories and hence files to be analyzed during testing.
• Generate a recursive directory of the phone file system and store it in a different computer for future reference.
• Install the mobile application and create a second recursive directory listing of the phone file system.
• Compare both the file system pre and post installation directory listing of the phone file systems.

Step-2 Fingerprinting: In step-1 our main was to verify the pre and post installation of files and directories. Now we would verify the contents of the files. The easiest way to do this is to create content hashes (MD5, SHA1, SHA 256 etc..) and compare them against various application runs. Use tools like md5deep2 to generate a MD5 hash of all files.

A command to create the recursive file hashes of the relevant directories on the phone system would look like:
md5deep –k –r k:\system k:\shared k:\resource k:\private k:\data >
First_FingerPrint.txt
This command recursively explores the directories and calculates the MD5 hash of all the files encountered.

User-Agent Request Header
Browser Based Mobile Application: A number of applications do not require to be installed in the mobile but rather can be accessed through mobile browsers. All web browsers include a User-Agent header in their requests. This header is used to identify the browser (IE, FF, Opera etc.) and also the device on which the browser is running.

Using a web proxy: Most web proxies (like Paros and Fiddler) provide the user with an option to modify a request header. In most cases this is a onetime configuration change and requires you to know the User-Agent header of the device being tested

Example:
User-Agent : NokiaE61i-1/3.0 (1.0633.22z.05) SymbianOS/9.1 Series60/3.0 Profile/MIDP-2.0 Configuration/CLDC-1.1
Configuring the web proxy to replace the existing User-Agent header to the one above may allow you to potentially access web applications supporting Nokia E61 phones from your desktop.

Firefox User Agent Switched add-on: This convenient add-on allows creating and storing of multiple arbitrary user agent headers. A mobile user agent profile can be activated to browse application just as you do from your mobile phone without the need for a proxy web server to change the headers.

Accept Request Header
All web browsers include a Accept header in their requests. This header is used to notify the server about the type of data the browser can accept. For example, web server can utilize the Accept header to determine whether the browser understands WML pages and thus return the appropriate content.
The WMLBrowser add-on in Firefox adds support to view pages and modifies the Accept header to reflect the same. The advantage of using such a tool is that once your desktop browser is configured, all testing can be performed as a regular web application penetration testing

Mobile Application Testing

2010-05-06T16:33:00.001+05:30

Challenges in Mobile Application Testing:

• Diversity of the Device Environment: The realm of mobile computing is composed of various types of mobile devices and underlying software (hundreds of device types, more than 40 mobile browsers). Some of the unique challenges involved in mobile testing as a result of this condition are:
I. Rendering of images and positioning of elements on the screen may be unsuitable in some devices due to the difference in display sizes across mobile devices and models. Exhaustive testing of user interfaces is necessary to ensure compatibility of the application.

II. Mobile devices have different application runtimes. Binary Runtime Environment for Wireless (BREW), Java, and embedded visual basic runtime are just some of the runtimes commonly available in mobile devices. Applications should be tested exhaustively for the variations specific to runtime.

• Hardware Configuration & Network-related Challenges: The mobile environment offers lesser memory and processing power for computing when compared with the traditional PC environment. Unlike the network landscape of the PC environment, the network landscape of a mobile device may have gateways (access points between the wireless internet and the cable internet). Some of the drawbacks of diverse hardware configurations and the network landscape of mobile devices are:

I. Limitations in processing speed and memory size of mobile devices lead to variations in performance of applications across different types of devices. Testing programs should ensure that the applications deliver optimum performance for all desired configurations of hardware.

II. Some devices communicate through WAP while some others use HTTP to communicate. Applications should be tested for their compatibility with WAP-enabled as well as HTTP-enabled devices.

III. Network latency (time taken for data transfer) will be unpredictable when applications communicate over network boundaries, leading to inconsistent data transfer speeds. Testing should measure the performance of applications for various network bandwidths.

IV. Gateways in a wireless network may act as data optimizers that deliver content more suitable for specific devices. This data optimization process may result in decreased performance for heavy traffic. Testing should determine the network traffic level at which gateway capabilities will impact the performance of the mobile application.

• Screens of various types: Different Mobile devices in market with all sort of shapes and sizes and all flashy looks make it difficult to exactly check the device compatibility of application.

• Keyboard versus stylus use: As we are talking here of Mobile Devices it just not consist of Mobile Phone while a big range of Mobile Devices is consisting of Personal Digital Assistance (PDA) as well as other smart devices .As PDA doesn’t consist of Keypad the things been done by using the touch screen functionality, so many a times designing and testing the applications for various such environment needed concern for such Keypad and stylus issues.

• Operating systems for mobile devices (Windows CE / Palm OS): Different operating system been loaded at different Mobile Client .The application should be well based at all the operating system and the uniformity and consistency of the application should not be lost out of running on different operating system.

• Proper internationalization: Normally we are not sure where the end user using our application is located, so it should be make a point with the Test Engineer that application doesn’t loose the motive out of running or accessing in other languages channel. The application should be in conformance with Unicode standards. Example it should display Chinese, Japanese and other characters properly.

• File formats: Mobile is just not specific to WML or CHTML, while nowadays it supports a lot more scripting language like XHTML and image formats like gif, jpeg other then wbmp and also j2me applications. So while testing the application we should make it a point to find out the support as per the device specification for all such file formats.

• File size: Different Mobile Devices comes up with different memory sizes and also their download capability varies according to that. We should make a point in our application that the downloadable image or game or any other thing should tune up with the Memory capabilities of respective device.

• Line and word-wrapping Features: Many a times using Label and Text Control we missed to take in account of screen size and display properties, although Pagination is a solution for this still we should take care of display specific issues in our application.

• Security issues: Mobile is just not a way of information sharing while e-commerce has a gateway to Mobile Phones. We have to look for the security issues when we are going to build transaction sites and other such applications. It should have proper planned session and cookie support as well WTLS and other security layer support for handling such transactions.

Guidelines for Testing Mobile Applications

1) Understand the network landscape and device landscape before testing to identify bottlenecks.
2) Conducting testing in uncontrolled real-world test conditions (field-based testing) is necessary, especially for a multi-tier mobile application.
3) Select the right automation test tool for the success of the testing program. Rules of thumb for an ideal tool are:
• One tool should support all desired platforms.
• The tool should support testing for various screen types, resolutions, and input mechanisms — such as touchpad and keypad.
• The tool should be connected to the external system to carry out end-to-end testing.

4) Use the Weighted Device Platform Matrix method to identify the most critical hardware/ platform combination to test. This method will be very useful especially when hardware/ platform combinations are high and time to test is low.
5) Check the end-to-end functional flow in all possible platforms at least once.
6) Conduct performance testing, GUI testing, and compatibility testing using actual devices. Even though these tests can be done using emulators, testing with actual devices is recommended.
7) Measure performance only in realistic conditions of wireless traffic and user load.

DATAWAREHOUSE TESTING

2009-04-06T14:48:00.002+05:30

There is an exponentially increasing cost associated with finding software defects later in the development lifecycle. In data warehousing, this is compounded because of the additional business costs of using incorrect data to make critical business decisions. Given the importance of early detection of software defects, let's first review some general goals of testing an ETL application:

* Data completeness : Ensures that all expected data is loaded.

* Data transformation : Ensures that all data is transformed correctly according to business rules and/or design specifications.

* Data quality : Ensures that the ETL application correctly rejects, substitutes default values, corrects or ignores and reports invalid data.

* Performance and scalability : Ensures that data loads and queries perform within expected time frames and that the technical architecture is scalable.

* Integration testing : Ensures that the ETL process functions well with other upstream and downstream processes.

* User-acceptance testing : Ensures the solution meets users' current expectations and anticipates their future expectations.

* Regression testing : Ensures existing functionality remains intact each time a new release of code is completed.

Data Completeness

One of the most basic tests of data completeness is to verify that all expected data loads into the data warehouse. This includes validating that all records, all fields and the full contents of each field are loaded. Strategies to consider include:

* Comparing record counts between source data, data loaded to the warehouse and rejected records.

* Comparing unique values of key fields between source data and data loaded to the warehouse. This is a valuable technique that points out a variety of possible data errors without doing a full validation on all fields.

* Utilizing a data profiling tool that shows the range and value distributions of fields in a data set. This can be used during testing and in production to compare source and target data sets and point out any data anomalies from source systems that may be missed even when the data movement is correct.

* Populating the full contents of each field to validate that no truncation occurs at any step in the process. For example, if the source data field is a string(30) make sure to test it with 30 characters.

* Testing the boundaries of each field to find any database limitations. For example, for a decimal(3) field include values of -99 and 999, and for date fields include the entire range of dates expected. Depending on the type of database and how it is indexed, it is possible that the range of values the database accepts is too small.

Data Transformation

Validating that data is transformed correctly based on business rules can be the most complex part of testing an ETL application with significant transformation logic. One typical method is to pick some sample records and "stare and compare" to validate data transformations manually. This can be useful but requires manual testing steps and testers who understand the ETL logic. A combination of automated data profiling and automated data movement validations is a better long-term strategy. Here are some simple automated data movement techniques:

* Create a spreadsheet of scenarios of input data and expected results and validate these with the business customer. This is a good requirements elicitation exercise during design and can also be used during testing.

* Create test data that includes all scenarios. Elicit the help of an ETL developer to automate the process of populating data sets with the scenario spreadsheet to allow for flexibility because scenarios will change.

* Utilize data profiling results to compare range and distribution of values in each field between source and target data.

* Validate correct processing of ETL-generated fields such as surrogate keys.

* Validate that data types in the warehouse are as specified in the design and/or the data model.

* Set up data scenarios that test referential integrity between tables. For example, what happens when the data contains foreign key values not in the parent table?

* Validate parent-to-child relationships in the data. Set up data scenarios that test how orphaned child records are handled.

Data Quality

For the purposes of this discussion, data quality is defined as "how the ETL system handles data rejection, substitution, correction and notification without modifying data." To ensure success in testing data quality, include as many data scenarios as possible. Typically, data quality rules are defined during design, for example:

* Reject the record if a certain decimal field has nonnumeric data.

* Substitute null if a certain decimal field has nonnumeric data.

* Validate and correct the state field if necessary based on the ZIP code.

* Compare product code to values in a lookup table, and if there is no match load anyway but report to users.

Depending on the data quality rules of the application being tested, scenarios to test might include null key values, duplicate records in source data and invalid data types in fields (e.g., alphabetic characters in a decimal field). Review the detailed test scenarios with business users and technical designers to ensure that all are on the same page. Data quality rules applied to the data will usually be invisible to the users once the application is in production; users will only see what's loaded to the database. For this reason, it is important to ensure that what is done with invalid data is reported to the users. These data quality reports present valuable data that sometimes reveals systematic issues with source data. In some cases, it may be beneficial to populate the "before" data in the database for users to view.

Performance and Scalability

As the volume of data in a data warehouse grows, ETL load times can be expected to increase, and performance of queries can be expected to degrade. This can be mitigated by having a solid technical architecture and good ETL design. The aim of the performance testing is to point out any potential weaknesses in the ETL design, such as reading a file multiple times or creating unnecessary intermediate files. The following strategies will help discover performance issues:

* Load the database with peak expected production volumes to ensure that this volume of data can be loaded by the ETL process within the agreed-upon window.

* Compare these ETL loading times to loads performed with a smaller amount of data to anticipate scalability issues. Compare the ETL processing times component by component to point out any areas of weakness.

* Monitor the timing of the reject process and consider how large volumes of rejected data will be handled.

* Perform simple and multiple join queries to validate query performance on large database volumes. Work with business users to develop sample queries and acceptable performance criteria for each query.

Integration Testing

Typically, system testing only includes testing within the ETL application. The endpoints for system testing are the input and output of the ETL code being tested. Integration testing shows how the application fits into the overall flow of all upstream and downstream applications. When creating integration test scenarios, consider how the overall process can break and focus on touchpoints between applications rather than within one application. Consider how process failures at each step would be handled and how data would be recovered or deleted if necessary.

Most issues found during integration testing are either data related to or resulting from false assumptions about the design of another application. Therefore, it is important to integration test with production-like data. Real production data is ideal, but depending on the contents of the data, there could be privacy or security concerns that require certain fields to be randomized before using it in a test environment. As always, don't forget the importance of good communication between the testing and design teams of all systems involved. To help bridge this communication gap, gather team members from all systems together to formulate test scenarios and discuss what could go wrong in production. Run the overall process from end to end in the same order and with the same dependencies as in production. Integration testing should be a combined effort and not the responsibility solely of the team testing the ETL application.

User-Acceptance Testing

The main reason for building a data warehouse application is to make data available to business users. Users know the data best, and their participation in the testing effort is a key component to the success of a data warehouse implementation. User-acceptance testing (UAT) typically focuses on data loaded to the data warehouse and any views that have been created on top of the tables, not the mechanics of how the ETL application works. Consider the following strategies:

* Use data that is either from production or as near to production data as possible. Users typically find issues once they see the "real" data, sometimes leading to design changes.

* Test database views comparing view contents to what is expected. It is important that users sign off and clearly understand how the views are created.

* Plan for the system test team to support users during UAT. The users will likely have questions about how the data is populated and need to understand details of how the ETL works.

* Consider how the users would require the data loaded during UAT and negotiate how often the data will be refreshed.

Regression Testing

Regression testing is revalidation of existing functionality with each new release of code. When building test cases, remember that they will likely be executed multiple times as new releases are created due to defect fixes, enhancements or upstream systems changes. Building automation during system testing will make the process of regression testing much smoother. Test cases should be prioritized by risk in order to help determine which need to be rerun for each new release. A simple but effective and efficient strategy to retest basic functionality is to store source data sets and results from successful runs of the code and compare new test results with previous runs. When doing a regression test, it is much quicker to compare results to a previous execution than to do an entire data validation again.

Taking these considerations into account during the design and testing portions of building a data warehouse will ensure that a quality product is produced and prevent costly mistakes from being discovered in production.

Metrices

2009-04-06T14:45:00.000+05:30

1) Schedule variance = (Actual time taken - Planned time) / Planned time * 100

2) Effort variance = (Actual effort - Planned Effort)/Planned effort * 100

3) Defect unearthing efficiency = 100 * STRs found in pass 1/ ( STRs found in pass 1 + STRs found in pass 2 but existing in pass 1) = Defect unearthing efficiency for pass 1.

Thus defect unearthing efficiency can be found for all passes.

Also defect unearthing efficiency in terms of field STRs could be found as follows:

= 100 * Total STRs found in Polaris for a release/ (Total STRs found in Polaris for a release + Field STRs for that release)

4) Test case efficiency = (Total STRs - STRs not mapped)/Total STRs * 100

5) Test case coverage = (Total Test cases - STRs that cannot be mapped to test cases)/ Total Test Cases * 100

PRODUCTIVITY METRICS

1. OUTPUT / INPUT OR

VALUE OF MATERIAL / COST OF PRODUCTION

Eg: Non Commented Software Source (NCSS) Per Engineer Per Month

NCSS per Person Month

NCSS per Function Point

NCSS can also be replaced by KLOC (Kilo Lines of Code)

QUALITY METRICS

1. DEFECTS PER KLOC

2. DEFECTS PER FUNCTION POINT

3. % OF MAINTENANCE / PROJECT EFFORT

4. POST RELEASE STABILITY

5. PREDELIVERY DEFECT REMOVAL EFFICIENCY

6. DELIVERED DEFECTS PER KLOC

7. (COST OF REWORK / TOTAL COST ) X 100

PROCESS IMPROVEMENT METRICS

1. PROCESS IMPROVEMENT OR OVERALL EFFICIENCY = RETURN ON INVESTMENT

2. % OF RECYCYLED CODE

3. % OF CALENDER TIME / PHASE

4. INCREASED RETURNS / INCREASED COST

5. % OF DEFECTS IN PHASE OF THIS YEAR / % OF DEFECTS IN PHASE OF LAST YEAR

6. DIFFICULTY COMPLEXITY = McCABE COMPLEXITY

7. TOTAL RECORDED PRE-DELIVERY DEFECTS / TOTAL FOUND DURING UAT

8. POST RELEASE DISCOVERED DEFECTS DENSITY OVER PERIOD

Testing for Error Code

2009-02-05T15:45:00.001+05:30

A common error that we can see during our search is the HTTP 404 Not Found.
Often we can see this error code with many details about web server and other components.
For Example:

Not Found

The requested URL /page.html was not found on this server.

Apache/2.2.3 (Unix) mod_ssl/2.2.3 OpenSSL/0.9.7g DAV/2 PHP/5.1.2 Server at localhost Port 80

This error message can be generated with the insertion of non-existing URL.

After the common message that shows a page not found, there are information about web server version, OS, modules and other products used.

This information can be very important both for OS and for applications during a penetration test but web server errors aren't the only ones useful in a security analysis.

So, we'll analyze the next occurrence that shows an abnormal behaviour:

Microsoft OLE DB Provider for ODBC Drivers (0x80004005)

[DBNETLIB][ConnectionOpen(Connect())] - SQL server does not exist or access denied

What's happened?

We'll proceed step by step!

For example, the 80004005 is a generic IIS error code which indicates that isn't possible to access a database.

In many cases we can see that this code is followed by the version of the database so, the pentester with this information can plan an appropriate strategy for the security test.

Microsoft OLE DB Provider for ODBC Drivers error '80004005'

[Microsoft][ODBC Access 97 ODBC driver Driver]General error Unable to open registry key 'DriverId'

The first example shows a connection error message obtained by SQL Server Database because the database server which linked into application is down or credentials don't allow access.

But it isn't the only information that we know, in fact in this way we have discovered the kind of operating system.

In this case we could verify if the web application permits change of variables value to connect to the database.

In the second case we can see a generic error in the same situation (we know the database version) but with a different error message and database server.

But in the end...It's the same thing!

And now, we do a practical example with a security test on web application that looses the link with the database server because there is badly written code (the next error message is caused by the application which can't resolve the database server name or when the variable value is modified) or other network problems.

For example, we have a database administration web portal which can be connected to db server after a log-on phase to realize query, create tables and modify database fields.

Well, during POST of credentials for the log-on phase meet this message that evidences the presence of a MySQL database server:

Microsoft OLE DB Provider for ODBC Drivers (0x80004005)

[MySQL][ODBC 3.51 Driver]Unknown MySQL server host

If we see in the HTML code of the log-on page the presence of a hidden field with a database IP, we can try to change this value in the URL with the address of another database (our database for example).

Another example: knowing the database server that services a web application, we can take advantage of this information to carry out a SQL Injection for that kind of database or a persistent XSS test.

Information Gathering on web applications with server side technology is quite difficult so, the information discovered can be useful for the correct execution of an attempted exploit and reduce false positives.

Black Box testing and example

Test:

telnet 80

GET / HTTP/1.1

Result:

HTTP/1.1 404 Not Found

Date: Sat, 04 Nov 2006 15:26:48 GMT

Server: Apache/2.2.3 (Unix) mod_ssl/2.2.3 OpenSSL/0.9.7g

Content-Length: 310

Connection: close

Content-Type: text/html; charset=iso-8859-1

Test:

1. network problems

2. bad configuration about host database address

Result:

Microsoft OLE DB Provider for ODBC Drivers (0x80004005) '

[MySQL][ODBC 3.51 Driver]Unknown MySQL server host

Test:

1. Authentication Failed

2. Credentials not inserted

Result:

Firewall version used for authentication

Error 407

FW-1 at : Unauthorized to access the document.

• Authorization is needed for FW-1.

• The authentication required by FW-1 is: unknown.

• Reason for failure of last attempt: no user

Automated Testing Methodology

2008-05-06T14:35:00.000+05:30

Automated Testing Methodology

Commonly Used Methodologies:

Record/Playback Method

· The Test Tool's Recording Mechanism records keystrokes, mouse actions, verification lists, etc.

Functional Decomposition Method

· The Application is broken down into Business Functions

· Automated Scripts are developed using the Tool's Scripting Language to perform those functions

· Data-Driven Process using Input & Verification Data Files

Test-Plan Driven Method

· The Test Cases are broken down into 'generic' testing actions

· Scripts are developed using the Tool's Scripting Language to perform these 'generic' actions

· Input File controls the processing as well as providing the Input and Verification data

Record/Playback Method

Advantages:

· Easy to Use

· Tester just starts recording and executes the Manual Test Case

Disadvantages:

· Reliability

The tester can make errors, which are then recorded, and have to be corrected

Failure on replay due to timing issues

Failure on replay due to events that occur that were not recorded (pop-ups, messages, etc.)

· Maintenance

Scripts contain hard-coded data that must be updated if the Application or the Data changes

Scripts must be enhanced or corrected after recording

Scripts & verification need to be re-recorded if the Application processing changes

Functional Decomposition Method

Business Functionality is broken down into its fundamental operations.

Test Case Example: "Post a Payment and Verify Account Data is Updated Correctly"

This could be broken down into the following operations:

Navigation: Access Payment Screen from Main Menu

Post a Payment

Verify Current Balance Updated

Navigation: Return to Main Menu

Navigation: Access Account Record

Verify Account Data Updated (Balance, Next Payment Date, etc.)

Navigation: Return to Main Menu

Using this breakdown, we can extract Business Functionality:

Navigation Routine(s)

Payment Posting & Verification Routine

Account Data Verification Routine

Routines can be Data-Driven, using Input and Verification files.

Numerous Test Cases can be automated by simply adding Input and Verification files for each.

Functional Decomposition Method

Architecture:

Driver Scripts

· Perform Initialization (if required), then call Test Case Scripts in the desired order

· Can be arranged to account for Test Case dependencies (If Test 1 fails, skip to Test 4, etc.)

Test Case Scripts

· Call Business Function Scripts to perform the required Test Case actions and verifications

Business Function Scripts

· Perform specific Business Functions within the application

· Call Subroutine Scripts and User-Defined Functions tp perform specific actions

Subroutine Scripts

· Perform application-specific tasks required by two or more Business Function Scripts

User-Defined Functions

Perform general, specific tasks that can be used by any number of scripts

Navigation, Enter Data using Input File, Verify Data using Verification File, etc.

Advantages:

Maintenance

· Modular design: If a Business Function changes, only 1 or 2 scripts must be modified

· Complex Test Cases can be constructed by calling Business Function Scripts from a Main Routine

· Data-Driven: Script can be used for many Test Cases by using different input/verification files

Reliability

· Tester error is eliminated, as is "scripter error" after the script has been properly coded and tested

· Unexpected events (pop-ups, messages, etc.) can be anticipated and coded for

Disadvantages:

Maintenance

· Each Business Function requires a script. There may be hundreds of Business Functions

· Changes in Test Cases require updates to several sets of input/verification files for each Test Case

· Format of input/verification records must be strictly adhered to or the tests will fail

· Testers must maintain the input/verification records as well as the Test Case documentation

Totally Data-Driven Method

Testing Activity is broken down into it's fundamental actions.

Examples: Data Entry

Select item in list_box/combo_box

Set a radio_button on or off

Set check_box on or off

Set a spin_control to a value

Enter text in an edit_box or field

Actions

Press a push_button or key

Select a menu_item

Select a tab

Verification

Verify correct Screen or Window is displayed

Verify Data (corollary to Data Entry)

Verify Field or Object Attributes

Each Testing Action is Associated with a Key-Word

Each Key-Word is Associated with a Utility Script

Data Entry Key-Word: "Enter:" Utility Script: Enter()

Action Key-Word: "Action:" Utility Script: Action()

Verify Window Key-Word: "Verify:" Utility Script: Verify()

Verify Data Key-Word: "Verify_Data:" Utility Script: Ver_Data()

Verify Attributes Key-Word: "Verify_Attributes:" Utility Script: Ver_Attr()

A Spreadsheet can be used for Input to this process:

Key-Words are placed in Column-1

Parameters or Field/Object names are placed in Column-2

Data or Field/Object names are placed in Column-3

Column 4 is used for comments

How it Works:

Spreadsheet is saved as a tab-delimited (text) file

A Controller script reads and processes the tab-delimited file

Switch/Case is used to match on the Column-1 Key-Word

A "list" is created from the remaining columns (Column-2~Column-3|Column-2~Column-3|etc.)

This continues until a blank line is reached

The Controller script then calls the Utility Script associated with the Key-Word, passing it the created list.

The Utility Script processes the input list from the Controller

Splits the input list into an array

Processes the Column-2 & Column-3 data in much the same way as the Controller Script

Returns to Controller script when finished

A Driver script processes multiple Test Cases, calling the Controller script

The File Name of the tab-delimited text file representing the Test Case is passed to the Controller

Architecture:

Driver Scripts

· Perform Initialization (if required), then call the Controller Script passing it the Test Case file name

· Also can be arranged to account for Test Case dependencies (If Test 1 fails, skip to Test 4, etc.)

Controller Script

· Calls Utility Scripts associated with Key-Words to perform the Test Case actions and verifications

Utility Scripts

· Perform specific Testing tasks required: Data Entry, Actions, Data Verification, etc.

· Call Subroutine Scripts and User-Defined Functions to perform specific actions

Business Function Utility Scripts

· Perform required application-specific tasks which may be a combination of Testing tasks

· Are associated with application-specific Key-Words and are parameterized within the Spreadsheet

User-Defined Functions

· Perform general tasks that can be used by any number of scripts

Architecture using TestDirector: TestDirector acts as the Driver

Single Test-Case Drivers call the Controller Script, passing the name of the tab-delimited

File Structure:

C:\

· AUT ------------- Application Under Test - Main Dir

· APP_Init ---------- --- Initialization Scripts

· DLL_Lib --------- --- DLLs to import (if required)

· Exp_Res --------- --- Common Expected Results Dir

· Fcn_Lib ----------- --- Application-Specific Functions

· GUI_Lib ---------- --- GUI Files for the Application

· Parameters ------ --- Parameter Files (set variables represented in the spreadsheet)

· Scripts ------------ --- Application-Specific Scripts

· AUT_Util ---- ---------- Application-Specific Utility Scripts

· Bus_Fcn ------ ---------- Application-Specific Business Function Scripts

· Drivers ------- ---------- Application-Specific Driver Scripts

· Recorded ---- ---------- Application-Specific Recorded Scripts

· TestData --------- --- Tab-Delimited Files

· TestPlan ---------- --- Spreadsheets / Workbooks

· TestRept --------- --- Customized Test Reports

· ToolKit_Fcn -- General ToolKit Function Libraries

· ToolKit_Util -- General Utility Scripts

Using TestDirector as the Driver:

To use Mercury Interactive's TestDirector, a slightly different architecture must be applied. TestDirector is designed to manage and call each different test or test case. Therefore we insert a Test Case Driver between TestDirector and the Controller Script:

TestDirector

· Calls the Test Case Driver script for each Test Case

· May also call a Test Case Script composed of Business Functions

· Additionally may call a recorded script, or any type of script

Test Case Driver

· Calls the Controller script, passing to it the Test Case name

· Test Case Driver folder contains the Spreadsheet file and the tab-delimited file

· Test Case Driver folder may also contain any other input/verification files required

From this point, the architecture is the same: The Controller script calls Utility / Business Function Scripts associated with Key-Words Utility Scripts perform the specific Testing Tasks delineated within the Spreadsheet

Business Function Scripts perform application-specific Testing Tasks User-defined functions perform generic tasks that can be used by any script as required.

Test Case Driver

Test Case Driver Folder contains all files & data relevant to the Test Case:

C:\ Mercury

AUT--------- Application Under Test - Main Dir

TestCases----- --- Test Case Driver Scripts (Called by TestDirector)

TC0001---- -------- Test Case Driver Script Folder

db (Folder)

exp (Folder)

Header (Test Properties)

Script (WinRunner TSL Script)

Script.bak (Script Backup Copy)

TC001.xls (Spreadsheet Test Case)

TC001.txt (Tab-Delimited File)

Results (Folder)

TC001R.txt (Customized Test Results)

Summary:

The Keyword-Driven method allows the Tester to create Test Cases in a Spreadsheet and have these be used as Automated Test Cases without having to use or learn the Test Tool. Automated Script development and maintenance is reduced, as only a few dozen scripts will be needed rather than hundreds. Test Case (Spreadsheet) maintenance is neither increased nor reduced, as the Tester must currently update the manual test cases anyhow when changes or revisions must be made. Only one "Test Tool Expert" will be required. The whole testing staff will not have to learn to use the test tool (unless they want to). Business Function Scripts and Recorded Scripts can still be used under this architecture if required. Parameterized input as variable-data allows Test Case (Spreadsheet) to be re-used in multiple-database environments. Pre-written "ToolKit" Functions and Utility Scripts greatly reduces "ramp-up" time.

Password Recovery Testing

2007-08-02T13:11:00.000+05:30

Whats is the Issue?

A great majority of web applications provide a way for users to recover (or reset) their password in case they have forgotten it. The exact procedure varies heavily among different applications, also depending on the required level of security, but the approach is always to use an alternate way of verifying the identity of the user. One of the simplest (and most common) approaches is to ask the user for his/her e-mail address, and send the old password (or a new one) to that address. This scheme is based on the assumption that the user's email has not been compromised and that is secure enough for this goal.
Alternatively (or in addition to that), the application could ask the user to answer one or more "secret questions", which are usually chosen by the user among a set of possible ones. The security of this scheme lies in the ability to provide a way for someone to identify themselves to the system with answers to questions that are not easily answerable via personal information lookups.

How To Test?

Password Reset
The first step is to check whether secret questions are used. Sending the password (or a password reset link) to the user email address without first asking for a secret question means relying 100% on the security of that email address, which is not suitable if the applicaton needs a high level of security.
On the other hand, if secret question are used, the next step is to assessing their strength.
As a first point, how many questions need to be answered before the password can be reset ? The majority of applications only need the user to answer to one question, but some critical applications require the user to answer correctly to two or even more different questions.
As a second step, we need to analyze the questions themselves. Often a self-reset system offers the choice of multiple questions; this is a good sign for the would-be attacker as this presents him/her with options. Ask yourself whether you could obtain answers to any or all of these questions via a simple Google search on the Internet or with some social engineering attack. As a penetration tester, here is a step-by-step walk through of assessing a password self-reset tool:

Are there multiple questions offered?

If so, try to pick a question which would have a “public” answer; for example, something Google would find with a simple query
Always pick questions which have a factual answer such as a “first school” or other facts which can be looked up
Look for questions which have few possible options such as “what make was your first car”; this question would present the attacker with a short-list of answers to guess at and based on statistics the attacker could rank answers from most to least likely

Determine how many guesses you have (if possible)

Does the password reset allow unlimited attempts ?
Is there a lockout period after X incorrect answers? Keep in mind that a lockout system can be a security problem in itself, as it can be exploited by an attacker to launch a Denial of Service against users

Pick the appropriate question based on analysis from above point, and do research to determine the most likely answers
How does the password-reset tool (once a successful answer to a question is found) behave?

Does it allow immediate change of the password?
Does it display the old password?
Does it email the password to some pre-defined email address?
The most insecure scenario here is if the password reset tool shows you the password; this gives the attacker the ability to log into the account, and unless the application provides information about the last login the victim would not know that his/her account has been compromised.
A less insecure scenario is if the password reset tool forces the user to immediately change his/her password. While not as stealthy as the first case, it allows the attacker to gain access and locks the real user out.
The best security is achieved if the password reset is done via an email to the address the user initially registered with, or some other email address; this forces the attacker to not only guess at which email account the password reset was sent to (unless the application tells that) but also to compromise that account in order to take control of the victim access to the application.

The key to successfully exploiting and bypassing a password self-reset is to find a question or set of questions which give the possibility of easily acquiring the answers. Always look for questions which can give you the greatest statistical chance of guessing the correct answer, if you are completely unsure of any of the answers. In the end, a password self-reset tool is only as strong as the weakest question. As a side note, if the application sends/visualizes the old password in cleartext it means that passwords are not stored in a hashed form, which is a security issue in itself already.

Password Remember

The "remember my password" mechanism can be implemented with one of the following methods:

Allowing the "cache password" feature in web browsers. Although not directly an application mechanism, this can and should be disabled.
Storing the password in a permanent cookie. The password must be hashed/encrypted and not sent in cleartext.

For the first method, check the HTML code of the login page to see whether browser caching of the passwords is disabled. The code for this will usually be along the following lines:

The password autocomplete should always be disabled, especially in sensitive applications, since an attacker, if able to access the browser cache, could easily obtain the password in cleartext (public computers are a very notable example of this attack). To check the second implementation type – examine the cookie stored by the application. Verify the credentials are not stored in cleartext, but are hashed. Examine the hashing mechanism: if it appears a common well-known one, check for its strength; in homegrown hash functions, attempt several usernames to check whether the hash function is easily guessable. Additionally, verify that the credentials are only sent during the login phase, and not sent together with every request to the application.

Security Testing: Session Hacking

2007-07-24T16:59:00.000+05:30

Security Testing: Session Hacking

Directory Traversal Attackes

2007-07-21T17:00:00.000+05:30

Properly controlling access to web content is crucial for running a secure web server. Directory Traversal is an HTTP exploit which allows attackers to access restricted directories and execute commands outside of the web server's root directory.Web servers provide two main levels of security mechanisms:

Access Control Lists (ACLs)

Root directory

An Access Control List is used in the authorization process. It is a list which the web server's administrator uses to indicate which users or groups are able to access, modify or execute particular files on the server, as well as other access rights.

The root directory is a specific directory on the server file system in which the users are confined. Users are not able to access anything above this root.

For example: the default root directory of IIS on Windows is C:\Inetpub\wwwroot and with this setup, a user does not have access to C:\Windows but has access to C:\Inetpub\wwwroot\news and any other directories and files under the root directory (provided that the user is authenticated via the ACLs).

The root directory prevents users from accessing sensitive files on the server such as cmd.exe on Windows platforms and the passwd file on Linux/UNIX platforms.This vulnerability can exist either in the web server software itself or in the web application code.

In order to perform a directory traversal attack, all an attacker needs is a web browser and some knowledge on where to blindly find any default files and directories on the system.What an attacker can do if your site is vulnerable. With a system vulnerable to Directory Traversal, an attacker can make use of this vulnerability to step out of the root directory and access other parts of the file system. This might give the attacker the ability to view restricted files, or even more dangerous, allowing the attacker to execute powerful commands on the web server which can lead to a full compromise of the system.

Depending on how the website access is set up, the attacker will execute commands by impersonating himself as the user which is associated with "the website". Therefore it all depends on what the website user has been given access to in the system.Example of a directory traversal attack via web application code

In order to perform a directory traversal attack, all an attacker needs is a web browser and some knowledge on where to blindly find any default files and directories on the system. Example of a directory traversal attack via web application codeIn web applications with dynamic pages, input is usually received from browsers through GET or POST request methods. Here is an example of a GET HTTP request URL:

http://test.XYZ.com/show.asp?view=oldarchive.html

With this URL, the browser requests the dynamic page show.asp from the server and with it also sends the parameter "view" with the value of "oldarchive.html". When this request is executed on the web server, show.asp retrieves the file oldarchive.htm from the server's file system, renders it and then sends it back to the browser which displays it to the user. The attacker would assume that show.asp can retrieve files from the file system and sends this custom URL:

http://test.xyz.com/show.asp?view=../../../../../Windows/system.ini

This will cause the dynamic page to retrieve the file system.ini from the file system and display it to the user. The expression ../ instructs the system to go one directory up which is commonly used as an operating system directive. The attacker has to guess how many directories he has to go up to find the Windows folder on the system, but this is easily done by trial and error. Example of a directory traversal attack via web server.

Apart from vulnerabilities in the code, even the web server itself can be open to directory traversal attacks. The problem can either be incorporated into the web server software or inside some sample script files left available on the server. The vulnerability has been fixed in the latest versions of web Server software, but there are web servers online which are still using older versions of IIS and Apache which might be open to directory traversal attacks. Even tough you might be using a web
Server software version that has fixed this vulnerability, you might still have some sensitive default script directories exposed which are well known to hackers. For example, a URL request which makes use of the scripts directory of IIS to traverse directories and execute a command can be:

http://server.com/scripts/..%5c../Windows/System32/cmd.exe?/c+dir+c:\

The request would return to the user a list of all files in the C:\ directory by executing the cmd.exe command shell file and run the command "dir c:\" in the shell. The %5c expression that is in the URL request is a web server escape code which is used to represent normal characters. In this case %5c represents the character "\".

Newer versions of modern web server software check for these escape codes and do not let them through. Some older versions however, do not filter out these codes in the root directory enforcer and will let the attackers execute such commands.

Session Hacking

2007-07-20T11:15:00.000+05:30

Technical security testing of Session Management implementation covers two key areas:

Integrity of Session ID creation
Secure management of active sessions and Session IDs

The Session ID should be sufficiently unpredictable and abstracted from any private information, and the Session management should be logically secured to prevent any manipulation or circumvention of application security. These two key areas are interdependent, but should be considered separately for a number of reasons. Firstly, the choice of underlying technology to provide the sessions is bewildering and can already include a large number of OTS products and an almost unlimited number of bespoke or proprietary implementations. Whilst the same technical analysis must be performed on each, established vendor solutions may require a slightly different testing approach, and existing security research may exist on the implementation. Secondly, even an unpredictable and abstract Session ID may be rendered completely ineffectual should the Session management be flawed. Similarly, a strong and secure session management implementation may be undermined by a poor Session ID implementation. Furthermore, the analyst should closely examine how (and if) the application uses the available Session management. It is not uncommon to see Microsoft ISS server ASP Session IDs passed religiously back and forth during interaction with an application, only to discover that these are not used by the application logic at all. It is therefore not correct to say that because an application is built on a ‘proven secure’ platform its Session Management is automatically secure.

Session Analysis

The Session Tokens (Cookie, SessionID or Hidden Field) themselves should be examined to ensure their quality from a security perspective. They should be tested against criteria such as their randomness, uniqueness, resistance to statistical and cryptographic analysis and information leakage.

Token Structure & Information Leakage

The first stage is to examine the structure and content of a Session ID provided by the application. A common mistake is to include specific data in the Token instead of issuing a generic value and referencing real data at the server side. If the Session ID is clear-text, the structure and pertinent data may be immediately obvious as the following:

192.168.100.100:user:password:25:51
If part or the entire Token appears to be
encoded or hashed, it should be compared to various techniques to check for
obvious obfuscation. For example the string 192.168.100.100:user:password:25:51
is represented in Hex, Base64 and as an MD5 hash:

Hex 3139322E3136382E3130302E313A6F77617370757365723A70617373776F72643A31353A3538Base64 MTkyLjE2OC4xMDAuMTpvd2FzcHVzZXI6cGFzc3dvcmQ6MTU6NTg=MD5 01c2fc4f0a817afd8366689bd29dd40aHaving identified the type of obfuscation, it may be possible to decode back to the original data. In most cases, however, this is unlikely. Even so, it may be useful to enumerate the encoding in place from the format of the message. Furthermore, if both the format and obfuscation technique can be deduced, automated brute-force attacks could be devised. Hybrid tokens may include information such as IP address or User ID together with an encoded portion,

Analysis of the variable areas (if any) of the Session ID should be undertaken to establish the existence of any recognizable or predictable patterns. These analysis may be performed manually and with bespoke or OTS statistical or cryptanalytic tools in order to deduce any patterns in Session ID content. Manual checks should include comparisons of Session IDs issued for the same login conditions – e.g. the same username, password and IP address. Time is an important factor which must also be controlled. High numbers of simultaneous connections should be made in order to gather samples in the same time window and keep that variable constant. Even a quantization of 50ms or less may be too coarse and a sample taken in this way may reveal time-based components that would otherwise be missed. Variable elements should be analysed over time to determine whether they are incremental in nature. Where they are incremental, patterns relating to absolute or elapsed time should be investigated. Many systems use time as a seed for their pseudo random elements. Where the patterns are seemingly random, one-way hashes of time or other environmental variations should be considered as a possibility. Typically, the result of a cryptographic hash is a decimal or hexadecimal number so should be identifiable. In analysing Session IDs sequences, patterns or cycles, static elements and client dependencies should all be considered as possible contributing elements to the structure and function of the application.

Are the Session IDs provably random in nature? e.g. Can the result be reproduced?
Do the same input conditions produce the same ID on a subsequent run?
Are the Session IDs provably resistant to statistical or cryptanalysis?
What elements of the Session IDs are time-linked?
What portions of the Session IDs are predictable?
Can the next ID be deduced even given full knowledge of the generation algorithm and previous IDs?

Brute Force Attacks
Brute force attacks inevitably lead on from questions relating to predictability and randomness. The variance within the Session IDs must be considered together with application session durations and timeouts. If the variation within the Session IDs is relatively small, and Session ID validity is long, the likelihood of a successful brute-force attack is much higher. A long session ID (or rather one with a great deal of variance) and a shorter validity period would make it far harder to succeed in a brute force attack.

How long would a brute-force attack on all possible Session IDs take?
Is the Session ID space large enough to prevent brute forcing? e.g. is the length of the key sufficient when compared to the valid life-span
Do delays between connection attempts with different Session IDs mitigate the risk of this attack?