Genesis is an application provisioning tool used to automate the creation of software and configuration packages. It is designed to simplify the delivery of large scale provisioning and deployment projects, with the objective of significantly reducing the time and money invested in them.
Genesis Code checked into SourceForge - Sun, 18 May 2008
The first cut of the Genesis code has been checked into SourceForge. We have now moved our source repository from dev2dev's codeshare site to SourceForge and migration of all Genesis artifacts (code, documentation and website) is now complete. (0 comments)
We are constantly updating our documentation so please check these pages regularly.
Getting Started for Linux OS
Getting Started for Windows OS
Developers Guide
MyApplication Configuration Package
Engine and Core API
Specification Documents
Configuration Hierarchy
Archetypes
Schemas
Coming Soon…
Coming Soon…
Genesis is a made available by the efforts of our members. While we try to attract as much sponsorship as possible the project remains independent and is community driven. We encourage active participation by individuals or groups who wish to contribute positively to the project. All support and involvement is most welcome.
Software and configuration packages come in many different flavours form source packages like Red Hats SRPMS to binary packages such as Suns Solaris PKG and Microsofts MSI. Configuration packages like those created with Debian’s config-package-dev utility and other application archives such as WAR’s and EAR’s bear testimony to proliferation and variety of package types. With this in mind its easy to see the value of a single tool and approach to create all these types of archives.
Genesis works on some basic assumptions that allows it create these types of packages. Namely that these archives generally code some of the file types listed below:
An archetype is a template of an applications configuration files. Generally any non binary or non content application artefact will be included in an application archetype. Application archetypes implement the designs, standards and specifications defined in an organisations standard build for the application and Genesis maintains archetypes for all the applications that exist in an enterprise stack.
To create a deployable software of configuration package, Genesis draws custom configuration data and combines it with the application archetype in a process called transcription. Since the same application archetype is used to build the packages for different environments there will be a high degree of standardisation across all the deployed instances of the application.
In order to maintain standardisation across the enterprise the best practice is to maintain one archetype for any single version of an application. In reality this may not be achievable, and you may find that you need to maintain two or three archetypes per application, however the more archetypes you use for a given application the less standardised the deployment of that application becomes. Under these circumstances you should review the causes of variation and make sure they are justified.
The process by which configuration data is combined with the application archetype is called transcription. Transcription produces an application configuration package (e.g. RPM, Tar ball, Zip file) of the application that can deployed by third party orchestration tools such as those provided by HP or BMC. The transcription process also conducts compliance testing on the package to ensure it is configured in accordance with organisational standards.
Genesis application archetypes are maintained as a product catalogue. Each archetype comes with a standard build document that details it’s design decisions, standards and specifications used to create it.
Every configurable hardware or software entity in a provisioning and deployment project is treated as an application in Genesis. That is to say that the configuration data and meta data used to build and install software, ranging from server operating systems to network device configurations, is created using the same process. These configurations are maintained in a series of flat files which are formatted in XML that conforms to the Genesis schema. The Genesis configuration hierarchy differentiates between logical and physical configurations and branches them appropriately.
The logical branch of a configuration describes the services, environments and applications deployed by the enterprise. Within every environment multiple applications may be used to implement a particular service. For example ACME Bank may have an on-line banking service deployed to development, UAT, integration and production environments. In the development environment a WebLogic server and Oracle database are used to implement the service, while on the production environment a complete suite of applications might be deployed.
The physical branch of a configuration describes the properties of a real objects. ACME banks physical configuration branch might describe multiple data centres with numerous racks installed with dozens of servers and network devices. When Genesis builds an application it combines logical and physical configurations and transcribes them with the application archetype. The result is an application that is configured for a logical environment located in a physical data centre.
The following instructions describe the steps involved in getting the Genesis demonstration application installed and operational. We used a server running Fedora 8 for this installation but in practice any GNU/Linux distribution is suitable. Genesis will also run under windows but this guide does not describe that installation. If you already have Java and Ant installed you may skip to section 4. Environment Set-up.
Genesis requires a Java Development Kit 1.6 and Apache Ant 1.7.0. You can download them from the links listed below.
Java Development Kit
Apache Ant
Genesis
Sun offers of choice of Java runtime packages for download. In this example we downloaded the jdk-6u6-linux-i586.bin file to our /tmp directory. After you have downloaded the file you should move it to a reasonable location and extract it. We moved the package to the /opt folder, made it executable and extracted it there. You should do this as the root user.
# Become root user
[user]$ su -# Move the package from /tmp to /opt
[root]# mv /tmp/jdk-6u6-linux-i586.bin /opt# Make the package executable
[root]# chmod +x /opt/jdk-6u6-linux-i586.bin# Extract the package
[root]# /opt/jdk-6u6-linux-i586.bin# Create a symbolic link to the jdk
[root]# ln -s /opt/jdk1.6.0_06/ jdkThe Apache Foundation provide a number of archive formats for Ant. In this example we used apache-ant-1.7.0-bin.tar.gz. After we had downloaded it we moved the archive to the /opt folder from /tmp and extracted it there. You should do this as the root user.
# Become root user
[user]$ su -# Move the package from /tmp to /opt
[root]# mv /tmp/apache-ant-1.7.0-bin.tar.gz /opt# Extract the package
[root]# tar zxvf /opt/apache-ant-1.7.0-bin.tar.gz# Create a symbolic link to the jre
[root]# ln -s /opt/apache-ant-1.7.0 apache-antBefore installing genesis we need to set up our environment to make sure Genesis has everything it needs. This entails setting up a genesis user and making sure the environment variables for JAVA_HOME and ANT_HOME are set-up correctly and exported.
# Become root user
[user]# su -# Add a genesis user and group
[root]# useradd -u 1010 genesis# Choose a secure password for this user
[root]# passwd genesis# Become the genesis user
[root]# su - genesisEdit the genesis users .bash_profile and add the JAVA_HOME and ANT_HOME environment variables. see the example .bash_profile below:
########################################################### # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs JAVA_HOME=/opt/jdk ANT_HOME=/opt/apache-ant PATH=$PATH:$HOME/bin/:$JAVA_HOME/bin:$ANT_HOME export PATH JAVA_HOME ANT_HOME ###########################################################
# Confirm the environment variables are correct:
[genesis]$ source .bash_profile
[genesis]$ echo $JAVA_HOME
/opt/jdk
[genesis]$ echo $ANT_HOME
/opt/apache-ant# Confirm Java and Ant versions
[genesis]$ java -fullversion
java full version "1.6.0_05-b13"
[genesis]$ ant -version
Apache Ant version 1.7.0 compiled on December 13 2006Once we have installed Java, Ant and our environment has been configured we are ready to install Genesis. We need to download it and extract it to a suitable directory.
# Become genesis user
[user]$ su - genesis# Make a dist directory in the genesis users home
[genesis]$ mkdir dist# Move the package from /tmp to /home/genesis/dist
[genesis]$ mv /tmp/genesis-{version}.tar.gz /home/genesis/dist# Extract the package
[genesis]$ tar zxvf /home/genesis/dist/genesis-{version}.tar.gz# Change directory to the genesis distribution
[genesis]$ cd /home/genesis/dist/genesis-{version}# Execute the genesis without any options to see usage.
[genesis]$./genesisXML schema’s have a number of uses in the XML world, these include validation, documentation, querying support, data binding and guided editing of XML files. Within Genesis schema’s fulfill two primary functions, these are:
Apart from keeping the Genesis engine free of validation code, leveraging XML schema for these purposes provides a robust, extensible and open standards compliant mechanism for ensuring the integrity of configuration hierarchy.
All Genesis XML Schema files (.xsd files) are stored under genesis/engine/schemas within the genesis distribution.
In order to maintain the integrity of the data stored in the configuration hierarchy Genesis schemas exist for all elements of the logical and physical configuration and are named after the elements they validate. Every application being provisioned by Genesis will also have it’s own schema. The schemas use a combination of predefined simple data types and derived simple and complex data types. These schemas maintain and enforce a standard and consistent structure and semantic for all the XML files stored in the configuration hierarchy.
The role of nomenclature in the automation and standardisation of infrastructure provisioning cannot be overstated. A simple, consistent and justifiable infrastructure nomenclature is the foundation which all automated processes are built upon, yet the benefits of such a nomenclature reach far beyond provisioning. Encoding and validating naming rules has traditionally been a time consuming and expensive activity which has limited the extent to which the rules are can be applied and enforced.
Genesis enforces infrastructure nomenclature by encoding the naming rules using regular expressions to define simple data types within XML schemas. This mechanism is incredibly powerful, new naming rules can be quickly implemented, existing rules can be changed and “what if” scenarios can be tested with ease against existing configurations.
The following schema files are compose validate and enforce nomenclature in the configuration hierarchy:
common.xsd
enterprise.xsd
service.xsd
environment.xsd
datacentre.xsd
rack.xsd
node.xsd
View the getting started guide for Genesis installation instructions.
Please read the release notes for the most up to date information about the package.
genesis genesis-0.0.34-tar.gz released (Sun, 04 Jan 2009 12:31:09 GMT) - Sun, 04 Jan 2009
Released at Sun, 04 Jan 2009 12:31:09 GMT by salimb
Includes files: genesis-dist-0.0.34.tar.gz (109449 bytes, 70 downloads to date)
[Download] [Release Notes]
genesis genesis-0.0.33.tar.gz released (Sun, 12 Oct 2008 16:49:55 GMT) - Sun, 12 Oct 2008
Released at Sun, 12 Oct 2008 16:49:55 GMT by salimb
Includes files: genesis-0.0.33.tar.gz (206680 bytes, 28 downloads to date)
[Download] [Release Notes]
These documents detail the design and configuration choices used for enterprise, environment and application specifications. The examples provided cover only basic implementations but serve a good example for starting your own specifications.
The following instructions describe the steps involved in building a Genesis distribution from source. We used a server running Fedora 8 for this installation but in practice any GNU/Linux distribution is suitable. Genesis will also run under windows but this guide does not describe that installation.
The Genesis Source uses Maven 2 (http://maven.apache.org) for generating a usable Genesis installation. Note however that Maven is not required to use Genesis. To install Maven, follow the instructions available from the apache software foundation. The Genesis source is arranged into a number of modules. Currently, these modules are:
GENESIS_SRC_ROOT/
ant-tasks/
dist/
model/
schemas/
ANT-TASKS
This module contains the ant tasks for genesis such as those for resolving genesis objects to file names, building filter sets, and iterating of Genesis objects.
DIST
This module contains all the sample files packaged into the distribution, along with all of the packaging configuration to produce the genesis-dist.zip file in the correct layout.
MODEL
This module contains all the code necessary to work with genesis.xml configuration files. It defines the underlying object model for Genesis, along with the mechanisms to load and query them.
SCHEMAS
This module contains the definitions for the XML schemas relevant to the genesis core, such as the schema for the genesis.xml file.
To generate a distributable Genesis binary, execute:
mvn packageYou must be in the GENESIS_SRC_ROOT directory. This will download all necessary dependencies, and build each of the modules. The output distributable can be found in the dist/target directory as a .zip file.
Written in Java, the Genesis engine and its core API is one of three key architectural components together with the configuration hierarchy and application archetypes.
The Genesis engine drives the creation of packages by using application archetypes. Archetypes, in their simplest incarnation, contain two files which detail the build instructions and the associated configuration data for a particular application. The build instructions are written in XML whilst the configuration file contains tokens which are replaced by values obtained from the configuration hierarchy. The resulting output files are copied to the build directory where they are packaged and ready deployment.
The process of creating archetypes and populating the configuration hierarchy is simple and intuitive. The Genesis engine is built upon and extends Apache Ant. This has number benefits for Genesis users:
During the creation of an application package, Genesis can execute defined tests to ensure the application configuration meets defined standards. For example, Genesis can check the validity of security certificates used by an application or test for correct networking configuration.
The following instructions describe the steps involved in getting the Genesis demonstration application installed and operational for Windows OS. If you already have Java and Ant installed you may skip to section 4. Environment Set-up.
Genesis requires a Java Development Kit 1.6 and Apache Ant 1.7.0. You can download them from the links listed below.
Buildfile: build.xml does not exist!
Build failed
This indicates a successful installation. Typing ant -version gives the version and the date it was compiled:
Apache Ant version 1.7.1 compiled on July 17 2008
Before installing genesis we need to set up our environment to make sure Genesis has everything it needs. This involves setting the correct Environment Variables for JAVA_HOME and ANT_HOME. Environment Variables are edited via:
Start> My Computer> View System Information> Advanced> Environment Variables
Once we have installed Java, Ant, and our environment has been configured we are ready to install Genesis. The Genesis genesis-{version}.tar.gz files can now be downloaded from the homepage and extracted to a suitable directory (e.g. C:\genesis-{version}). To test the installation navigate to the genesis root and execute the genesis.cmd to show usage.
We can view the test archetype for MyApplication in the genesis-{version}
genesisarchetypesMyApplication directory. The archetype appears as follows:
# Static properties
CommonProperty=@Common.Name@
EnterpriseProperty=@Enterprise.Name@
# Logical Properties
ServiceProperty=@Service.Name@
EnvironmentProperty=@Environment.Name@
ApplicationProperty=@Application.Name@
# Physical Properties
DatacentreProperty=@Datacentre.Name@
RackProperty=@Rack.Name@
ServerProperty=@Server.Name@
Open the command prompt and navigate to the Genesis-{version}genesis directory. Run the build tool, this time specifying the archetype property values:
genesis.cmd MyEnterprise MyDatacentre MyRack MyServer MyService MyEnvironment MyApplication
A successful build creates a new directory in the Genesis root called build with the following directory structure:
buildMyEnterpriseMyServiceMyEnvironmentMyDatacentreMyRackMyServerMyApplication
The new MyApplication.cfg file which holds the Static, Logical, and Physical properties specified for the application configuration package is copied to the MyApplication folder, and appears as follows:
# Static properties
CommonProperty=CommonProperty
EnterpriseProperty=MyEnterprise
# Logical Properties
ServiceProperty=MyService
EnvironmentProperty=MyEnvironment
ApplicationProperty=MyApplication
# Physical Properties
DatacentreProperty=MyDatacentre
RackProperty=MyRack
ServerProperty=MyNode
Genesis © 2008