Tuesday, August 25, 2009

Just because

Look, just as I am leaving this blog and am not using it does not mean that I will allow it to be a Splog.

To the SCUM who keep (attempted) posting crud regarding their crap products - I will be blocking and filtering ALL comments as usual.

Sploggers are LOSERS!

Monday, July 27, 2009

Where to now?

The exam is looming and the assignments are submitted, so it is back to the former blog.

I will be adding a number of posts on both the SANS forensic Blog as well as my own in the next few days. I am leaving this material live on the web (even after the close of the session) and I hope that it is of use.

Assessment item 2

This page links to the various sub-pages and posts associated with Assignment No.2.

The second Assignment comprises of 14 exercises:
Ruby on Rails Workshops 5 to 8
And the second Elevator Pitch.
Where needed, there are also additions to Assignment 1.

CASE (Computer Aided Software Engineering) Tools

Case tools can be a great aid to auditing database systems. CASE or Computer Assisted Software Engineering tools not only help in the development of software and database structures but can be used to reverse engineer existing databases and check them against a predefined schema. There are a variety of both open source and commercial CASE tools. In this chapter we’ll be looking at Xcase (http://www.xcase.com/).

Many commercial databases can run into the gigabyte or terabyte in size. Standard command line SQL coding is unlikely to find all of the intricate relationships between these tables, stored procedures and other database functions. A CASE tool on the other hand can reverse engineer existing databases to produce diagrams that represent the database. These can first of all be compared with existing schema diagrams to ensure that the database matches the architecture that it is originally built from and to be able to quickly zoom in on selected areas.

Visual objects, colors and better diagrams may all be introduced to further enhance the auditor’s capacity to analyze the structure. Reverse engineering a database will enable the auditor to find out the various structures that have been created within the database. Some of these include:
  • The indexes,
  • Fields,
  • Relationships,
  • Sub-categories,
  • Views,
  • Connections,
  • Primary keys and alternate keys,
  • Triggers,
  • Constraints,
  • Procedures and functions,
  • Rules,
  • Table space and storage details associated with the database,
  • Sequences used and finally the entities within the database.
Each of the tables will also display detailed information concerning the structure of each of the fields that may be viewed at a single glance. In large databases a graphical view is probably the only method that will adequately determine if relationships between different tables and functions within a database actually meet the requirements. It may be possible in smaller databases to determine the referential integrity constraints between different fields, but in a larger database containing thousands of tables there is no way to do this in a simple manner using manual techniques.
Fig- Display database schema.
When reviewing the database design, it is not just security functions such as cross site scripting and sequel injection that need to be considered. Relationships between various entities and the rights and associated privileges that are associated with various tables and roles also need to be considered. The CASE tools allow us to visualize the most important security features associated with a database. These are:
  1. Schemas restrict the views of the database for users,
  2. Domains, assertions, checks and other integrity controls defined as database objects which may be enforced using the DBMS in the process of database queries and updates,
  3. Authorization rules. These are rules which identify the users and roles associated with the database and may be used to restrict the actions that a user can take against any of the database features such as tables or individual fields,
  4. Authentication schemes. These are schemes which can be used to identify users attempting to gain access to the database or individual features within the database.
  5. User defined procedures which may define constraints or limitations on the use of the database,
  6. Encryption processes. Many compliance regimes call for the encryption of selected data on the database. Most modern databases include encryption processes that can be used to ensure that the data is protected.
  7. Other features such as backup, check point capabilities and journaling help to ensure recovery processes for the database. These controls aid in database availability and integrity, two of the three legs of security.
CASE tools also contain other functions that are useful when auditing a database. One function that is extremely useful is model comparison.
Fig - Reverse Engineer existing databases into presentation quality diagrams in minutes.

Case tools allow the developer to:
  • Present clear data models at various levels of detail using visual objects, colors and embedded diagrams to organize database schemas,
  • Synchronize models with the database,
  • Compare a baseline model to the actual database (or to another model),
Case tools can generate code automatically and also store this for review and baselining. This includes:
  • DDL Code to build and change the database structure
  • Triggers and Stored Procedures to safeguard data integrity
  • Views and Queries to extract data
The auditor can also document the database design using multiple reporting options. This allows for the printing of diagrams and reports and the addition of comments to the reports and user defined attributes to the model.

Data management features allow the auditor to validate the data in the database being reviewed against the business rules and constraints defined in the model and generate detailed integrity reports. This can be extended further to access and edit the data relationally using automatic parent/child browsers and lookups and then to locate faulty data subsets using automatically generated SQL statements. These provide valuable sources of errors and help in database maintenance – making the audit all the more valuable.

Model comparison involves comparing the model of the database with the actual database on the system. This can be used to ensure change control or to ensure that no unauthorized changes have been made for other purposes. To do this, a baseline of the database structure will be taken at some point in time. At a later time the database could be reverse engineered to create another model and these two models could be compared. Any differences, variations or discrepancies between these would represent a change. Any changes should be authorized changes and if not, should be investigated. Many of the tools also have functions that provide detailed reports of all discrepancies.

Many modern databases run into the terabytes and contain tens of thousands of tables. A baseline and automated report of any differences, variations or discrepancies makes the job of auditing change on these databases much simpler. Triggers and stored procedures can be stored within the CASE tool itself. These can be used to safeguard data integrity. Selected areas within the database can be set up such as honeytoken styled fields or views that can be checked against a hash at different times to ensure that no-one has altered any of these areas of the database. Further in database tables it should not change. Tables of hashes may be maintained and validated using the offline model that has stored these hash functions already. Any variation would be reported in the discrepancy report.

Next the capability to create a complex ERD or Entity Relationship Diagram in itself adds value to the audit. Many organizations do not have a detailed structure of the database and these are grown organically over time with many of the original designers having left the organization. In this event it is not uncommon for the organization to have no idea about the various tables that they have on their own database.

Another benefit of CASE tools is their ability to migrate data. CASE tools have the ability to create detailed SQL statements and to replicate through reverse engineering the data structures. They can then migrate these data structures to a separate database. This is useful as the data can be copied to another system. That system may be used to interrogate tables without fear of damaging the data. In particular the data that has migrated to the tables does not need to be the actual data, meaning that the auditor does not have access to sensitive information but will know the defenses and protections associated with the database. This is useful as the auditor can then perform complex interrogations of the database that may result in damage to the database if it was running on the large system. This provides a capability for the auditor to validate the data in the database against the business rules and constraints that have been defined by the models and generate detailed integrity reports. This capability gives an organization advanced tools that will help them locate faulty data subsets through the use of automatically generated SQL statements.

UML and Mapping Processes

Unified Modeling Language (UML) is a visual representation language designed for the purpose of modeling and communicating the information contained within systems. To do this it uses a series of diagrams and supporting text.

It can provide details of many process fields such as the following:
  • Actors, examples could include a manager leading a team executing a project and staff members on the project team.
  • The various processes that occur.
  • Relationships between actors and entities.
In UML, unified came about due to the Object Management Group (OMG) and Rational Software Corporation coming together to create an industry standard for engineering practices. This was a desire to create a common language.

A model is a depiction of a subject. A model is used to encapsulate a set of ideas (called abstractions) concerning a subject. A model provides a simple means to create a common understanding amongst different team members and other individuals. This helps create an understanding of the requirements of the system and to communicate the impact of changes that will occur to the system through development and use.

The creation of a model should be done in stages. An attempt to create a model all in one go is likely to become overwhelming. This may be possible and small systems, but large systems with many thousands of tables are beyond the human capacity to comprehend at once.

When modeling, good practice dictates that the auditor will capture the relevant information that is required to gain an understanding of the problem at hand. This information may then be used to solve problems are issues that have arisen and will aid in the recommendation of a solution. It is also necessary to exclude information that is not relevant to the task at hand. It is easy to be waylaid by immaterial facts that can in no way lead to a change in the system or are not related to the scope of a project.

In order to effectively manage the overall complexity involved within the audit of complex systems such as mainframes, models are an effective tool to achieve our goal. This process is best completed through:
  • Managing the abstractions that make up the model,
  • Including enough detail to understand the abstraction but not so much as to sidetrack the audit,
  • Exclude irrelevant information, and
  • Work with multiple teams to ensure that the model is relevant.
A language enables both people and systems to communicate about a subject. The subject incorporates the requirements and the system with respect to system development and audit. Language simplifies the process of communicating between individual team members and allows for the successful completion of the project.

Languages are not always composed of words. In fact, complex abstractions such as mathematics are in fact languages.

UML is formally defined by its creators as a language for specifying, visualizing, constructing, and documenting the artifacts of a system-intensive process. This is a system-intensive process used as an approach that centers on a system. It includes the various stages used to both produce and maintain a system. This is based on the requirements needed by the system. The specification includes the creation of a model describing the system. This model simplifies the analysis of the system and allows even complex systems to be audited within a reasonable timeframe and scope.

This process involves visualization through the use of diagrams designed to render the model into a simple form so that it can be communicated. This diagram is then an expression of the system. It could be likened to a blueprint for a building. Ideally, his blueprint is designed before the building, but like many system design projects, development of a model or blueprint has either been excluded or lost. The subsequent creation of this model through audit captures a baseline that can be used not only to understand the process at hand but also for use in future reviews and assessments. Documenting these systems captures the knowledge and requirements associated with the system.

UML and Processes
UML is not a process, it as a tool to capture processes and system design. A process relates a series of stages that are illustrated through the use of a methodology in order to decipher an issue. It then enables the development of a system that is designed to satisfy the requirements of a system owner or users.

Method addresses the following stages of the development process:
  • Requirements or information gathering,
  • analysis, and
  • design.
Methodology addresses the entire development process starting with the requirements or information gathering through to the system being made live.

The distinct means of collecting and using requirements, analyzing requirements and finally designing a system are the techniques utilized. Artifacts are the “work products” produced and used within a process. These include the documentation and the actual system.

Each classification of UML diagram is known as a modeling technique.
The use of a UML diagram (as depicted in the figure above) can greatly simplify the audit process for complex systems.

Further information about UML
The following sites are the principal sources for information about the UML standard:
  • The Object Management Group (OMG),
o http://www.omg.org and http://www.omg.org/uml
  • Rational Software Corporation (IBM),
o http://www.rational.com and http://www.rational.com/uml

The subsequent sites present information concerning the next major change to the UML (the OCL) and a variety of other information on the subject:
  • The Object Constraint Language (OCL),
o http://www.klasse.nl/ocl/index.html
  • The UML Forum is a virtual community concerning the UML,
o http://www.uml-forum.com
  • The Cetus Team provides UML tools, methodologies and processes,
o http://www.cetus-links.org