Cyclone Server Side Deployment Sytem

This is the server part of the Cyclone deployment system.
 * Contains drivers and HDD images
 * Database
 * Web based log (info from DB)
 * Creation of on-the-fly menu's for Syslinux (info from DB)
 * Postscript tempplate for auto label printing

=Folder structure=

The cyclone/ root folder can be anywhere, everything is relative to it. (I have it as a root folder as the server is dedicated for running it.)

=CSS= Simple bit of CSS style sheet for the log and index page and any other pages you may wish with info on them.

This is the way they are referenced (fonts pulled from google)  

=DB sqlite3 database=

The single database contains all the user configurable items for the HDD images and menu. It is broken down into 8 tables. See below link for full details about the database.
 * The full SQLite 3 database contents is here


 * Note the www server group (www-data) needs read/write access to the DB folder AND the actual database file

$ ls -al db total 1084 drwxrwxr-x 2 clone www-data   4096 2012-02-24 10:26. drwxr-xr-x 13 clone clone     4096 2012-02-15 09:31 .. -rw-rw-r-- 1 clone www-data 606208 2012-02-24 10:26 cyclone.sqlite

=DP/ Driverpack extraction and .inf processing=

Simple script that extracts the driverpacks from the downloaded .7z into their folders, maintaining the same structure.

Any existing extracted packs are removed, except the special /CYC folder (e.g. cyclone/DPwin7/DPx/x64/CYC) that contains your custom/additional drivers.

If you have just added a custom driver to /CYC, select just "Process (i)ds only" from the menu, to re-create the DP-$os-$i.list pack<>id map.

Extraction
Extraction depends mainly on your CPU and a little on I/O, as a guide on this server (see specs here) takes about 10 mins for XP.

INF processing
In order to perform the driverpack matching during imaging, all the inf's are examined and a txt file (DP-xxx-xxx.list) is built up relating the hardware to the relative path the driver is in. Any added custom drivers in the DPxxx/CYC folder are also processed along with the standard DP's. Currently searched for are
 * VEN/DEV (PCI)
 * VID/PID (USB)
 * ACPIxxxxx (ACPI)
 * TOSxxxxx (Toshiba)
 * HPxxxxxx (HP)
 * PNPxxxxx (some wierd and wonderful things on laptops)

This process takes half a minute min and needs to be run anytime the driverpacks are changed. It has no bearing on imaging speed. Quite some time was spent trying to get the info processing as fast as possible!

e.g.

During imaging this list is compared to one created by the client system and the folders containing the matched drivers are copied across. Note, this could be migrated to use the DB but the speed of this matching this way is near instant, adding DB processing would only add server load and slow it down.

=DPwin7=

Downloaded packs should be put in the appropriate root folders e.g. cyclone/system/DPwin7/

They are extracted/processed by the cyclone/DP/DP-extract.sh script which creates the sub folders and the PCI/USB ID mapping file e.g. /cyclone/DPwin7/DPx/DP-win7-x64.list

At time of writing the packs look like this;

$ ls -l /cyclone/DPwin7/ total 924232 -rw-r--r-- 1 clone clone    21870 2011-11-11 10:00 DP_AMDfilter_wnt6-x64_1110.7z -rw-r--r-- 1 clone clone 189987308 2011-11-11 09:56 DP_Audio_wnt6-x64_1111.7z  -rw-r--r-- 1 clone clone 10226022 2011-11-11 10:26 DP_Webcam_wnt6-x64_1103.7z -rw-r--r-- 1 clone clone 13678715 2011-07-27 09:03 DP_WLAN_wnt6-x64_1104.7z drwxr-xr-x 3 clone clone     4096 2012-02-24 11:55 DPx

$ ls -l /cyclone/DPwin7/DPx/ total 228 -rw-r--r-- 1 clone clone 226923 2012-02-23 10:11 DP-win7-x64.list drwxr-xr-x 13 clone clone  4096 2010-04-12 08:21 x64

$ ls -l /cyclone/DPwin7/DPx/x64/ total 88 drwxr-xr-x 5 clone clone  4096 2011-03-07 21:01 3 drwxr-xr-x 5 clone clone  4096 2012-01-15 08:16 All drwxr-xr-x 3 clone clone  4096 2011-10-09 08:19 C -rwxr-xr-x  1 clone clone  9829 2012-02-05 12:11 Chipset_x64.txt drwxrwxr-x 11 clone clone 4096 2012-01-17 11:52 CYC  drwxr-xr-x 10 clone clone 4096 2010-07-24 11:47 W drwxr-xr-x  5 clone clone  4096 2012-01-15 08:18 Win7

A special CYC/ folder for manually putting extra drivers exists e.g. /cyclone/DPwin7/DPx/x64/CYC, I suggest using a similar structure when organising this.

$ ls -l /cyclone/DPwin7/DPx/x64/CYC total 36 drwxrwxr-x 5 clone clone 4096 2012-01-17 11:53 BCOM drwxrwxr-x 3 clone clone 4096 2011-11-17 12:03 HP_BUT drwxrwxr-x 3 clone clone 4096 2011-11-11 14:29 HP_DG drwxrwxr-x 3 clone clone 4096 2011-11-18 08:06 HP_KEYB drwxrwxr-x 3 clone clone 4096 2011-11-16 08:56 HP_VS drwxrwxr-x 2 clone clone 4096 2011-12-15 13:42 INTEL_HECI drwxrwxr-x 2 clone clone 4096 2011-12-15 13:44 INTEL_SOL drwxrwxr-x 4 clone clone 4096 2011-11-11 14:21 INTEL_WIF drwxrwxr-x 2 clone clone 4096 2011-12-06 16:06 TOS_BT

=DPwinxp= $ ls -l /cyclone/DPwinxp/ total 1075176 -rw-r--r-- 1 clone clone 74313034 2010-06-29 12:56 DP_Bluetooth_wnt5_x86-32_910.7z -rw-r--r-- 1 clone clone  4836040 2012-02-16 08:34 DP_Chipset_wnt5_x86-32_1201.7z  -rw-r--r-- 1 clone clone 78059723 2012-02-24 12:27 DP_WebCam_wnt5_x86-32_1107.7z -rw-r--r-- 1 clone clone 30485022 2012-02-24 12:14 DP_WLAN_wnt5_x86-32_1202.7z drwxr-xr-x 3 clone clone     4096 2012-02-24 12:25 DPx

$ ls -l /cyclone/DPwinxp/DPx/ total 444 drwxr-xr-x 10 clone clone  4096 2010-02-06 20:35 D -rw-r--r--  1 clone clone 249886 2012-02-24 12:27 DP-winxp-D.list -rw-r--r-- 1 clone clone   1185 2009-10-29 14:53 DriverPack_Bluetooth_wnt5_x86-32.ini -rw-r--r-- 1 clone clone   1068 2012-01-29 16:39 DriverPack_Chipset_wnt5_x86-32.ini  -rw-r--r-- 1 clone clone    236 2011-07-17 22:02 DriverPack_WebCam_wnt5_x86-32.ini -rw-r--r-- 1 clone clone    288 2012-02-13 16:44 DriverPack_WLAN_wnt5_x86-32.ini

$ ls -l /cyclone/DPwinxp/DPx/D total 32 drwxr-xr-x  7 clone clone 4096 2011-04-22 06:00 3 drwxr-xr-x 28 clone clone 4096 2012-01-29 11:59 C drwxrwxr-x  12 clone clone 4096 2012-02-21 10:21 CYC  drwxr-xr-x 73 clone clone 4096 2012-02-07 07:02 W

Special driver folder CYC/ $ ls -l /cyclone/DPwinxp/DPx/D/CYC total 48 drwxr-xr-x 3 clone clone 4096 2011-11-21 16:01 1 drwxr-xr-x 2 clone clone 4096 2011-11-21 16:01 2 drwxr-xr-x 3 clone clone 4096 2011-11-21 16:01 3  drwxr-xr-x 2 clone clone 4096 2011-11-21 15:59 RI drwxrwxr-x 2 clone clone 12288 2012-02-21 10:22 SYN

Random example of the driver match list /cyclone/DPwinxp/DPx/DP-winxp-D.list 04F2:B1C7#D/3/W/A0 04F2:B1CE#D/3/W/V11 04F2:B1CE#D/3/W/V12 04F2:B1CF#D/3/W/Sm 04F2:B1D0#D/3/W/A0 04F2:B1DA#D/3/W/V14 04F2:B1DA#D/3/W/V9 04f2:b1E5#D/3/W/SC1 04F2:B1EA#D/3/W/V11 04F2:B1FC#D/3/W/R0

=fonts= Contains any fonts you wish to use, either referenced in the style sheet or used by PHP/GD library generating menu pages on the fly.

=img (HDD imgages)= Contains all the HDD images,
 * low bandwidth(<=100Mbit) suggest using smallest images with .xz [bandwidth limiting]
 * high bandwidth (gigabit) larger but faster decompress images .gz [HDD write speed limiting]


 * For absolute speed and consistancy use machine specific images with as much as possible pre-installed. (To deployed OS booting in ~2 minutes)
 * For ultimate flexibility use a 'generic' image created on a VM (Virtualbox) and driverpack injection. (Seconds slower, but will need additional few mins while drivers install). This lets you image a machine blind without knowing what drivers it may need beforehand. Very useful if you deal with laptops!

=images= Repository for all the images/graphics used in the system, web pages, menu, Linux kernel ppm etc.

=PHP=
 * 1) Generates the Syslinux menus from the DB, and the graphics/text all on the fly. Needs php compiled with GD library. (cyclone.php)
 * 2) Outputs the log on the log page (log-show.php)
 * 3) Pokes data back into the log (cyclone-post.php)

Menus - cyclone.php
About half the code here is to generate the graphics and isn't really necessary if you want something that is just functional. (Part of the reason was an experiment in the practicalities of on-the-fly generated graphical menus for syslinux/pxelinux as no one seems to have done much with it, or rather I couldnt find anything and did this from scratch)

It is somewhat complicated with three things merged together as a lot of the code is the same, I hope the commenting is good enough!

Show log
";

// all done // destroy database object unset($db); ?>

 

Add to log
=system= This entire folder gets copied to the machine being imaged. Some of the paths here may look a bit odd it is just to make their final copying simpler.

system/ cyclone.sh

cyclone.sh the main script that runs on the client to control the whole imaging cycle See here for full details of the system (Client Side Deployment System)

BASE
This is mostly a finisher for Win XP, it sorts out driver conflicts using the DP ini files which are part of the packs. This is the system as put together by driverpacks.net team :)

devcon.exe  DP_Install_Tool.cmd  DPInst.xml    DPsFnshr.exe  DSPdsblr.exe  mute.exe     ROE.exe DevPath.exe DPINST.exe           DPs_BASE.exe  DPsFnshr.ini  makePNF.exe   pmtimer.exe  setup.exe

mbr
Contains Win XP/Win 7 boot records. MBR code is less important than you may think - win7 and server 2008 use came code. Syslinux MBR could replace all of them.

 ...

win7-mbr-446 winxp-mbr-446 server2008-mbr-446

ps
PostScript label source template for label printing ( ps/uol-label.ps). This gets modified on the fly to create a machine label that is printed out, containing things like machine name, MAC, serial number etc. It also encodes this data into a 3D QR barcode purely in Postscript, thanks to super clever Terry Burton for this bit! The idea is if you wish to quickly audit a room of machines you can go around with a scanner and zap them all quickly to get a completely accurate data set.

This is the small snippet of the PostScript code, the HHH,SSS,MMM etc are replaced with various information gathered from the machine/dmidecode data.

reg
Windows registry files to inject after imaging. Currently only used for winXP. cddb.reg winxp-RunOnceEx.reg
 * RunOnce, adds key to RunOnceEx to begin the Windows portion of the setup (join to domain, install software etc.)
 * CriticalDeviceDatabase (CDDB), injects MassStorage drivers into the CDDB and service sections of the registry, allowing it to boot without the dreaded 0x7b stop error! This also lets AHCI mode work in XP without problem.

sysprep
WinXP sysprep

$ tree /cyclone/system/sysprep /cyclone/system/sysprep ├── i386 │   └── $OEM$ │      └── CMDLINES.TXT └── sysprep.inf

sysprep.inf

 * the basic WinXP sysprep file, modified on the fly during imaging to alter HAL etc.
 * HAL only gets 'try and update to...' (UpdateHAL=) from default (standard HAL) if it needs to be.

[Unattended] OemSkipEula=Yes InstallFilesPath=C:\sysprep\i386 ;UpdateHAL= UpdateInstalledDrivers = Yes DriverSigningPolicy = Ignore NonDriverSigningPolicy = Ignore OemPreinstall=Yes
 * richud.com

[GuiUnattended] AdminPassword=* EncryptedAdminPassword=NO AutoLogon=Yes AutoLogonCount=1 OEMSkipRegional=1 TimeZone=85 OemSkipWelcome=1

[UserData] ProductKey=xxxxx-xxxxx-xxxxx-xxxxx-xxxxx FullName="*" OrgName="*" ComputerName=*

[Display] AutoConfirm=1

[RegionalSettings] Language=00000809

[Identification] JoinWorkgroup=WORKGROUP

[Networking] InstallDefaultComponents=Yes

cmdlines.txt
[COMMANDS] "C:\sysprep\sysprep -clean"
 * anything needing to run as XP boots prior to most other things, currently just for PnP cleanup
 * remember there is no command shell running, so any .batch script etc needs cmd /c running before it.

win7
Win7 post deployment driver and software scripts
 * Installs software that changes a lot more than the image (Flash etc.)

$ tree /cyclone/system/win7 /cyclone/system/win7 ├── cyclone-win7.bat ├── dpinst64.exe ├── DPINST.exe ├── dpinst.xml └── software ├── AdbeRdr1012_en_US.exe ├── CertMgr.Exe ├── install_flash_player_ax_64bit.exe ├── lerootca.der ├── savw_97_sa_sfx.exe ├── sophos-97-silent.exe ├── sw_lic_full_installer.exe └── win7-eduroam.xml

dpinst.xml
This will install all 'present' hardware. Windows uses its PnP installer to choose best match if mutliple DP's support same hardware.

     <suppressEulaPage/> <scanHardware/> <legacyMode/> <subDirectory>*</subDirectory> </dpInst>

Windows
Confusingly named, was left this way so it easily copied over from cyclone.sh keeping same path structure as what it overwrites.

Win7 sysprep file (unattend.xml)

$ tree /cyclone/system/Windows /cyclone/system/Windows └── System32 └── sysprep └── unattend.xml

unattend.xml
<ComputerName>*</ComputerName> modified on the fly by cyclone.sh post imaging, passwords or anything else could be too.

winxp
Windows XP post deployment files, replacement kernels.

$ ls /cyclone/system/winxp AdFind.exe  cyclone-winxp.bat  grep.exe      halacpi.dll  hal.dll       hwcheck.bat  hwcntrl.exe  NETUSER.EXE  ntkrnlmp.exe  sleep.exe Au3Info.exe devcon.exe         halaacpi.dll  halapic.dll  halmacpi.dll  hwcntrl.au3  netdom.exe   newsid.exe   sed.exe       tr.exe

hwcheck.bat
This is called several times during the post image process. It relies on status of newdev.dll to detect what is going on. Relies on external program tr.exe for output formatting and sleep.exe for sleep. Depending on how its called it can ;
 * Cause a delay until hardware starts to be installed
 * Cause a delay while hardware is being installed

hwcntrl.au3
This AutoIT script(aka Hardware Controller script) is compiled to hwcntrl.exe. Has been around several years as you can see and occasionally tweaked if anything new found. It should catch everything you are likely to come across.

It serves to do several things;
 * supress dialog boxes
 * go through any wizards that may pop up
 * catch some wierdness various installs may throw up