Cyclone Server Side Deployment Sytem

From richud.com
Jump to navigation Jump to search


This is the server part of the Cyclone deployment system.

  • Contains drivers and HDD images
  • Database
  • Web based log (info from DB)
  • Creation of on-the-fly menu's for Syslinux (info from DB)
  • Postscript tempplate for auto label printing

Folder structure

The cyclone/ root folder can be anywhere, everything is relative to it. (I have it as a root folder as the server is dedicated for running it.)

cyclone/ css/ CSS style sheets for log web page
db/ Database for images, menus, schema, log
DP/ Driverpacks extraction/processing script
DPwin7/ DPx/ x64/ Windows 7 64 bit Driverpacks
x32/ Windows 7 32 bit Driverpacks (not used)
DPwinxp/ DPx/ D/ Windows XP 32 bit Driverpacks
fonts/ Fonts for syslinux GD rendered menu
img/ Partition image files for WinXP/Win7 etc
images/ Pictures for log web page and syslinux menu
php/ PHP scripts for system, menu and log data post
system/ base/ Driverpacks BASE system files (XP)
mbr/ XP/Win7 MBR's
ps/ PostScript label source for label printing
reg/ Xp/Win7 registry adjustments (RunOnce/CriticalDeviceDatabase)
sysprep/ i386/ $OEM$/ WinXP sysprep files
win7/ software/ Win7 post deployment driver and software scripts
Windows/ System32/ sysprep/ Win7 sysprep files
winxp/ Win XP post deployment driver and software scripts

CSS

Simple bit of CSS style sheet for the log and index page and any other pages you may wish with info on them.

This is the way they are referenced (fonts pulled from google) 

<link REL=stylesheet TYPE=text/css HREF=../css/cyclone.css>
<link href=http://fonts.googleapis.com/css?family=Droid+Sans rel=stylesheet type=text/css> 

table {
width:90%;
border-top:1px solid #ff0000;
border-right:1px dashed #e5eff8;
margin:1em auto;
border-collapse:collapse;
border-spacing:10px;
font-smoothing:always;
}
td {
font:bold 0.7em 'Droid Sans', arial, sans-serif;
color:#678197;
border-bottom:1px dashed #e5eff8;
border-left:1px solid #ff0000;
border-spacing:10px;
padding:.3em 1em;
text-align:left;
margin-left: 10px;
margin-right: 10px;
margin-top: 5px;
margin-bottom: 5px;
}
td.t{
color:#ffffff;
}

thead{
text-align:left;
font:bold 1.5em 'Droid Sans', arial, sans-serif;
color:#a8a3d3;
}

h1 {
font:bold 1.4em 'Droid Sans', arial, sans-serif;
color:#678197
}

A {
font:bold 1.4em 'Droid Sans', arial, sans-serif;
color:#678197
}

h2 {
font:bold 1.6em 'Droid Sans', arial, sans-serif;
color:#FFFFFF
}

body { background-color:#06042A;color: #06042A;
font-smoothing:always;
}

DB sqlite3 database

The single database contains all the user configurable items for the HDD images and menu. It is broken down into 8 tables. See below link for full details about the database.

  • Note the www server group (www-data) needs read/write access to the DB folder AND the actual database file
$ ls -al db
total 1084
drwxrwxr-x  2 clone www-data   4096 2012-02-24 10:26 .
drwxr-xr-x 13 clone clone      4096 2012-02-15 09:31 ..
-rw-rw-r--  1 clone www-data 606208 2012-02-24 10:26 cyclone.sqlite

DP/ Driverpack extraction and .inf processing

Simple script that extracts the driverpacks from the downloaded .7z into their folders, maintaining the same structure.

Any existing extracted packs are removed, except the special /CYC folder (e.g. cyclone/DPwin7/DPx/x64/CYC) that contains your custom/additional drivers.

If you have just added a custom driver to /CYC, select just "Process (i)ds only" from the menu, to re-create the DP-$os-$i.list pack<>id map. 



#!/bin/bash

#extract packs
extract() {
	for i in $dppath
		do 
			#cleanup, be very careful with path as find output going to rm -rf !
			[[ -d "$i/" ]] && { rm *.ini 2>/dev/null ; find "$i/" -maxdepth 1 -mindepth 1 -type d ! -name "CYC" -exec rm -rf {} + ; }
		done

	echo "Extracting packs from  DP$os/*.7z to DP$os/DPx/"
	7za x "../*.7z" -y -xr'!x??/Server*' -xr'!x??/Vista*' -ssc- > /dev/null #skips uneeded drivers with new NT6 pack format
}


#extract ids
getids() {
	for i in $dppath
		do
			[[ -d "$i/" ]]  && echo "Processing infs in $i" || { echo "Skipping $i"; continue; }

			#cleanup temp
			tmp="/tmp/DP-$os-$i-extract.txt" ; [[ -e "$tmp" ]] && rm "$tmp"

			find "$i/" -type f -iname *.inf | while read j
				do
				echo "Processing $j"
					cat "$j" | tr -cd '[:alnum:]\r\n-=&_\\*'\
					 | grep -Ei '[V][EI][ND]_[A-Z0-9]{4}&|=.*\*[A-Z0-9]{7,8}|.*ACPI\\[A-Z0-9]{7,8}'\
					 | sort -u \
					 | grep -Eio '[V][EI][ND]_[A-Z0-9]{4}&[DP][EI][VD]_[A-Z0-9]{4}|\*[a-zA-Z0-9]{7,8}|\\[A-Z0-9]{7,8}'\
					 | sed "s,$,#${j%/*}," >> "$tmp"
	 			done
			chmod -R a+rx "$i"
			#fixup output, strip asterix and slashes , slashes from HPQ astertix from Tos
			cat "$tmp" | tr -d ' \\*' | tr '&' ':' | sed -e  's/[Vv][EeIi][NnDd]_\|[DdPp][EeIi][VvDd]_//g' | sort -u > "DP-$os-$i.list"
		done
}


#############
## Main Choices ##
#############
read -p "Choice 1/2, Please choose: Windows (x)P or Windows (7)" os
case "$os" in 
	x) 
		dppath="D"
		os=winxp
	;; 
	7) 
		dppath="x86 x64"
		os=win7 
	;; 
	*)
		echo "Bad choices, quitting" ; exit
	;;
esac


read -p "Choice 2/2, Please choose: (e)xtract packs & process ids OR e(x)tract packs ONLY OR process (i)ds only" act
[[ -d "../DP$os/DPx/" ]] || mkdir "../DP$os/DPx/"
pushd "../DP$os/DPx/"
case "$act" in 
	e) 
		extract
		getids
	;; 
	x) 
		extract
	;; 
	i) 
		getids
	;; 
	*)
		echo "Bad choices, quitting" ; exit
	;;
esac
popd
echo "Finished"

Extraction

Extraction depends mainly on your CPU and a little on I/O, as a guide on this server (see specs here) takes about 10 mins for XP.

INF processing

In order to perform the driverpack matching during imaging, all the inf's are examined and a txt file (DP-xxx-xxx.list) is built up relating the hardware to the relative path the driver is in. Any added custom drivers in the DPxxx/CYC folder are also processed along with the standard DP's. Currently searched for are

  • VEN/DEV (PCI)
  • VID/PID (USB)
  • ACPIxxxxx (ACPI)
  • TOSxxxxx (Toshiba)
  • HPxxxxxx (HP)
  • PNPxxxxx (some wierd and wonderful things on laptops)

This process takes half a minute min and needs to be run anytime the driverpacks are changed. It has no bearing on imaging speed. Quite some time was spent trying to get the info processing as fast as possible!

e.g.

During imaging this list is compared to one created by the client system and the folders containing the matched drivers are copied across. Note, this could be migrated to use the DB but the speed of this matching this way is near instant, adding DB processing would only add server load and slow it down.

DPwin7

Downloaded packs should be put in the appropriate root folders e.g. cyclone/system/DPwin7/

They are extracted/processed by the cyclone/DP/DP-extract.sh script which creates the sub folders and the PCI/USB ID mapping file e.g. /cyclone/DPwin7/DPx/DP-win7-x64.list

At time of writing the packs look like this; 

$ ls -l /cyclone/DPwin7/
total 924232
-rw-r--r-- 1 clone clone     21870 2011-11-11 10:00 DP_AMDfilter_wnt6-x64_1110.7z
-rw-r--r-- 1 clone clone 189987308 2011-11-11 09:56 DP_Audio_wnt6-x64_1111.7z
<SNIP>
-rw-r--r-- 1 clone clone  10226022 2011-11-11 10:26 DP_Webcam_wnt6-x64_1103.7z
-rw-r--r-- 1 clone clone  13678715 2011-07-27 09:03 DP_WLAN_wnt6-x64_1104.7z
drwxr-xr-x 3 clone clone      4096 2012-02-24 11:55 DPx

$ ls -l /cyclone/DPwin7/DPx/
total 228
-rw-r--r--  1 clone clone 226923 2012-02-23 10:11 DP-win7-x64.list
drwxr-xr-x 13 clone clone   4096 2010-04-12 08:21 x64

$ ls -l /cyclone/DPwin7/DPx/x64/
total 88
drwxr-xr-x  5 clone clone  4096 2011-03-07 21:01 3
drwxr-xr-x  5 clone clone  4096 2012-01-15 08:16 All
drwxr-xr-x  3 clone clone  4096 2011-10-09 08:19 C
-rwxr-xr-x  1 clone clone  9829 2012-02-05 12:11 Chipset_x64.txt
drwxrwxr-x 11 clone clone  4096 2012-01-17 11:52 CYC
<SNIP>
drwxr-xr-x 10 clone clone  4096 2010-07-24 11:47 W
drwxr-xr-x  5 clone clone  4096 2012-01-15 08:18 Win7

A special CYC/ folder for manually putting extra drivers exists e.g. /cyclone/DPwin7/DPx/x64/CYC , I suggest using a similar structure when organising this. 

$ ls -l /cyclone/DPwin7/DPx/x64/CYC
total 36
drwxrwxr-x 5 clone clone 4096 2012-01-17 11:53 BCOM
drwxrwxr-x 3 clone clone 4096 2011-11-17 12:03 HP_BUT
drwxrwxr-x 3 clone clone 4096 2011-11-11 14:29 HP_DG
drwxrwxr-x 3 clone clone 4096 2011-11-18 08:06 HP_KEYB
drwxrwxr-x 3 clone clone 4096 2011-11-16 08:56 HP_VS
drwxrwxr-x 2 clone clone 4096 2011-12-15 13:42 INTEL_HECI
drwxrwxr-x 2 clone clone 4096 2011-12-15 13:44 INTEL_SOL
drwxrwxr-x 4 clone clone 4096 2011-11-11 14:21 INTEL_WIF
drwxrwxr-x 2 clone clone 4096 2011-12-06 16:06 TOS_BT

DPwinxp

$ ls -l /cyclone/DPwinxp/
total 1075176
-rw-r--r-- 1 clone clone  74313034 2010-06-29 12:56 DP_Bluetooth_wnt5_x86-32_910.7z
-rw-r--r-- 1 clone clone   4836040 2012-02-16 08:34 DP_Chipset_wnt5_x86-32_1201.7z
<SNIP>
-rw-r--r-- 1 clone clone  78059723 2012-02-24 12:27 DP_WebCam_wnt5_x86-32_1107.7z
-rw-r--r-- 1 clone clone  30485022 2012-02-24 12:14 DP_WLAN_wnt5_x86-32_1202.7z
drwxr-xr-x 3 clone clone      4096 2012-02-24 12:25 DPx

$ ls -l /cyclone/DPwinxp/DPx/
total 444
drwxr-xr-x 10 clone clone   4096 2010-02-06 20:35 D
-rw-r--r--  1 clone clone 249886 2012-02-24 12:27 DP-winxp-D.list
-rw-r--r--  1 clone clone   1185 2009-10-29 14:53 DriverPack_Bluetooth_wnt5_x86-32.ini
-rw-r--r--  1 clone clone   1068 2012-01-29 16:39 DriverPack_Chipset_wnt5_x86-32.ini
<SNIP>
-rw-r--r--  1 clone clone    236 2011-07-17 22:02 DriverPack_WebCam_wnt5_x86-32.ini
-rw-r--r--  1 clone clone    288 2012-02-13 16:44 DriverPack_WLAN_wnt5_x86-32.ini

$ ls -l /cyclone/DPwinxp/DPx/D
total 32
drwxr-xr-x   7 clone clone 4096 2011-04-22 06:00 3
drwxr-xr-x  28 clone clone 4096 2012-01-29 11:59 C
drwxrwxr-x  12 clone clone 4096 2012-02-21 10:21 CYC
<SNIP>
drwxr-xr-x  73 clone clone 4096 2012-02-07 07:02 W

Special driver folder CYC/ 

$ ls -l /cyclone/DPwinxp/DPx/D/CYC
total 48
drwxr-xr-x 3 clone clone  4096 2011-11-21 16:01 1
drwxr-xr-x 2 clone clone  4096 2011-11-21 16:01 2
drwxr-xr-x 3 clone clone  4096 2011-11-21 16:01 3
<SNIP>
drwxr-xr-x 2 clone clone  4096 2011-11-21 15:59 RI
drwxrwxr-x 2 clone clone 12288 2012-02-21 10:22 SYN

Random example of the driver match list /cyclone/DPwinxp/DPx/DP-winxp-D.list 

04F2:B1C7#D/3/W/A0
04F2:B1CE#D/3/W/V11
04F2:B1CE#D/3/W/V12
04F2:B1CF#D/3/W/Sm
04F2:B1D0#D/3/W/A0
04F2:B1DA#D/3/W/V14
04F2:B1DA#D/3/W/V9
04f2:b1E5#D/3/W/SC1
04F2:B1EA#D/3/W/V11
04F2:B1FC#D/3/W/R0

fonts

Contains any fonts you wish to use, either referenced in the style sheet or used by PHP/GD library generating menu pages on the fly.

img (HDD imgages)

Contains all the HDD images,

  • low bandwidth(<=100Mbit) suggest using smallest images with .xz [bandwidth limiting]
  • high bandwidth (gigabit) larger but faster decompress images .gz [HDD write speed limiting]
  • For absolute speed and consistancy use machine specific images with as much as possible pre-installed. (To deployed OS booting in ~2 minutes)
  • For ultimate flexibility use a 'generic' image created on a VM (Virtualbox) and driverpack injection. (Seconds slower, but will need additional few mins while drivers install). This lets you image a machine blind without knowing what drivers it may need beforehand. Very useful if you deal with laptops!

images

Repository for all the images/graphics used in the system, web pages, menu, Linux kernel ppm etc.

PHP

  1. Generates the Syslinux menus from the DB, and the graphics/text all on the fly. Needs php compiled with GD library. (cyclone.php)
  2. Outputs the log on the log page (log-show.php)
  3. Pokes data back into the log (cyclone-post.php)

Menus - cyclone.php

About half the code here is to generate the graphics and isn't really necessary if you want something that is just functional. (Part of the reason was an experiment in the practicalities of on-the-fly generated graphical menus for syslinux/pxelinux as no one seems to have done much with it, or rather I couldnt find anything and did this from scratch)

It is somewhat complicated with three things merged together as a lot of the code is the same, I hope the commenting is good enough! 

<?php
/*
This has 3 functions
1) create a pxelinux menu.cfg automatically from the database
2) create a graphical overlay Cyclone menu via modification of a menubackground PNG to represent above
3) create Cyclone Options text menu
default location: /cyclone/cyclone.php
*/

$corefile="/rmdc/isolinux";

$db = new SQLite3("../db/cyclone.sqlite");

//graphical starting points
$x=570;
$y=86;
//main background image to build from
if(isset($_GET["img"])){
	$img=imagecreatefrompng("../images/cyclone.png");
}

//limit $l results per page
$l=11;

//get page offset - if more pages than $l
if(isset($_GET["offset"])){
	$o=$_GET["offset"];
	$otxt="&offset=$o";
} else {
	$o=0;
}

//create array of menu types to form Cyclone Options menu
$result = $db->query("SELECT * from menu") or die("\nError in query $q\n"); 
while($row = $result->fetchArray(SQLITE3_ASSOC)) { 
	$menuray[]=$row;
}


//CREATE TOP of MENU CFG FILE for "Cyclone Options" main Cyclone menu
//if no menu specified retrurn the one from row 0
$menu = isset($_GET["menu"]) ? $_GET["menu"] : $menuray[0]["menu"];
if(isset($_GET["cfg"])){ 
	echo "
	MENU BACKGROUND  /cyclone/php/cyclone.php?img=1$otxt&menu=$menu
	DEFAULT $corefile/vesamenu.c32

	MENU COLOR border	30;44      #00000000 #00000000 none

	MENU WIDTH 75
	MENU MARGIN 0
	MENU ROWS 17
	MENU TABMSGROW 21
	MENU CMDLINEROW 22
	MENU ENDROW 25

	
	Label Back to Main Menu
		MENU INDENT 25
		kernel $corefile/vesamenu.c32
	TEXT HELP
		Return to main menu
	ENDTEXT
	MENU SEPARATOR
	";
}

//CREATE TOP of MENU CFG file for Cyclone Options menu
if(isset($_GET["cfgmenu"])){
	echo "
	MENU BACKGROUND  /cyclone/images/cyclone.png
	DEFAULT $corefile/vesamenu.c32
	MENU COLOR border	30;44      #00000000 #00000000 none
	MENU WIDTH 68
	MENU MARGIN 15
	MENU ROWS 17
	MENU TABMSGROW 21
	MENU CMDLINEROW 22
	MENU ENDROW 25

	Label  Back to Main Menu
		kernel $corefile/vesamenu.c32
	TEXT HELP
		Return to main menu
	ENDTEXT
	";
}

//TOP MENU graphical - has to be as text is colourless
if(isset($_GET["img"])){
	$back=imagecreatefrompng("../images/back.png");
	list($w, $h) = getimagesize("../images/back.png");
	imagecopyresampled($img,$back,$x-200,$y-30,0,0,$w,$h,$w,$h);
}


//CREATE BODY of MENU CFG FILE
//find sql query for specified menu
foreach($menuray as $k=>$v){
	if($v["menu"]==$_GET["menu"]){$q=$v["sql"];$menutxt=$v["menuname"];break;}
}
//return matching sql query or if none, the query from row 0
$q = isset($q) ? $q : $menuray[0]["sql"];


//this is odd to get around fact there is no numrows facility in sqlite3/php.
if(!empty($o)){
	$q.=" LIMIT -1 OFFSET $o";
	$max=$o+$l;
} else {
	//dont set limit in SQL here as otherwise wont know if more results exist
	$max=$l;
}


$result = $db->query($q) or die("\nError in query $q\n"); 
//reset $yy
$yy=$y;
while($row = $result->fetchArray(SQLITE3_ASSOC)) {
//reset xx
$xx=$x;
	foreach ($row as $key=>$val){	
		$$key = $val;
			//create background img if set
			if(isset($_GET["img"])){
				if(file_exists("../images/$val.png")){
					//echo "png exists v $val vv $$val k $key kk $$key\n";
					$$val=imagecreatefrompng("../images/$val.png");
					list($w, $h) = getimagesize("../images/$val.png");
					imagecopyresampled($img,$$val,$xx,$yy,0,0,$w,$h,$w,$h);
					//shift back 50 pixels
					$xx-=50;
				}
				//echo "k $key kk $$key v $val vv $$val \n";
				//if($key=="scheme"){imagettftext($img, 10, 0, $xx-185,$yy+15, "0x10FFFFFF", "../fonts/Verdana.ttf", "$val");}

			}

	}
	//shift down 32 pixels (16 one line 32 two lines)
	$yy+=16;

	if(isset($_GET["cfg"])){
		echo  "
		LABEL $scheme
		LINUX $kernel $append spath=$spath runinit=$runinit serveraddr=$serveraddr snum=$snum imgmode=$imgmode label=$label
		INITRD $initrd
		TEXT HELP
			 [$scheme] [$imgmode] [$label]
		ENDTEXT
		#MENU SEPARATOR
		";
	}

	//increment row count / offset
	$o++;
	
	//if reached limit, output a next page menu entry
	if($o==$max){
		if(isset($_GET["img"])){
				$next=imagecreatefrompng("../images/next.png");
				list($w, $h) = getimagesize("../images/next.png");
				imagecopyresampled($img,$next,$x-20,$yy,0,0,$w,$h,$w,$h);
				$previous=imagecreatefrompng("../images/previous.png");
				list($w, $h) = getimagesize("../images/previous.png");
				imagecopyresampled($img,$previous,$x-20,$yy+=16,0,0,$w,$h,$w,$h);
		}
		if(isset($_GET["cfg"])){
			echo  "
			LABEL Next Page
			KERNEL $corefile/vesamenu.c32
			APPEND /cyclone/php/cyclone.php?cfg=1&menu=$menu&offset=$o
			TEXT HELP
				Next Page
			ENDTEXT
			LABEL Previous Page
			KERNEL $corefile/vesamenu.c32
			APPEND /cyclone/php/cyclone.php?cfg=1&menu=$menu&offset=".max(0,($o-$l-$l))."
			TEXT HELP
				Previous Page
			ENDTEXT
			";
		}
		break;
	}

}

//Write out background image
if(isset($_GET["img"])){
	//font size and crude scaling offsetting x from screen right
	$tz=32;
	$tx=$x-strlen($menutxt)*$tz*0.7;
	imagettftext($img, $tz, 0, $tx, 465, "0x70FFFFFF", "../fonts/Verdana.ttf", "$menutxt");
	header("Content-type: image/png");
	imagepng($img);
	imagedestroy($img);
}

//3
//CREATE Cyclone Options MENU CFG FILE
if(isset($_GET["cfgmenu"])){
	echo "MENU SEPARATOR";
	foreach ($menuray as $row){
		foreach ($row as $key=>$val){	
			$$key = $val;
		}
		echo "
		LABEL -> $menuname
		KERNEL $corefile/vesamenu.c32
		APPEND /cyclone/php/cyclone.php?cfg=1&menu=$menu
		";
	}
}

//DESTROP DB OBJECT
unset($db); 

?>

Show log

<html>
<head>
<title>Cyclone Log</title>
<link REL=stylesheet TYPE=text/css HREF=../css/cyclone.css>
<link href=http://fonts.googleapis.com/css?family=Droid+Sans rel=stylesheet type=text/css> 
</head>
<body>


<img align=right src=../images/lighthttpd.png border=0><img align=right src=../images/sqlite.png border=0>

<div align=center><h2>Cyclone Log File</h2></div>

<FORM action=<?php echo $_SERVER['PHP_SELF']; ?> method=post>
<select name="query">
  <option value="SELECT date,imgtime,product,serialnum,mac,hostname from log ORDER by date DESC LIMIT 25">Last 25</option>
  <option value="SELECT * from log ORDER by date DESC LIMIT 25">Last 25 Advanced</option>
  <option value="SELECT date,imgtime,product,serialnum,mac,hostname from log ORDER by date DESC LIMIT 50">Last 50</option>
  <option value="SELECT * from log ORDER by date DESC LIMIT 50">Last 50 Advanced</option>
  <option value="SELECT date,imgtime,product,serialnum,mac,hostname from log ORDER by date DESC LIMIT 100">Last 100</option>
  <option value="SELECT * from log ORDER by date DESC LIMIT 100">Last 100 Advanced</option>
  <option value="SELECT date,imgtime,product,serialnum,mac,hostname from log ORDER by date DESC">ALL</option>
  <option value="SELECT * from log ORDER by date DESC">ALL Advanced</option>
</select>
<input type="submit" name="Submit" value="Submit"> 
</FORM>

<?php

// /log/index.php
$db = new SQLite3("../db/cyclone.sqlite");


	$q = isset($_POST["query"]) ? $_POST["query"] : "SELECT date,imgtime,product,serialnum,mac,hostname from log ORDER by date DESC LIMIT 20";
	$result = $db->query($q) or die("Error in query"); 

	echo "<table border=1 class=m width=95%>"; 
	$i=0;$headers="";$cols="";
	while($row = $result->fetchArray(SQLITE3_ASSOC)) {
		foreach ($row as $key=>$val){
			if ($i==0) { $headers.="<td class=t>$key</td>"; }
			$cols.="<td>$val</td>"; 
		}
		if ($i==0) { echo "<tr>$headers</tr>\n";}
        	echo "<tr>$cols</tr>\n";
		$i++;$headers="";$cols="";
	} 
	echo "</table>";  



// all done 
// destroy database object 
unset($db); 
?>


<img align=right src=../images/lighthttpd.png border=0>
</BODY>

Add to log

<?php
/*
query DB: ../db/cyclone.db
return: text string and carriage return for each result
default file location: /cyclone/cyclone-post.php
*/


//debug
//print_r($_POST);


if (isset($_POST["q"])) {
	$q=stripslashes($_POST["q"]);
	$db = new SQLite3("../db/cyclone.sqlite");
	$result=$db->query($q)or die("Error in query\r\n"); 
	if (strpos($q,"INSERT")===false) {
		while($row = $result->fetchArray(SQLITE3_ASSOC)) { 
			foreach($row as $k=>$v){
				echo "$k=\"$v\"\n";
			}
		}
	}
	unset($db); 
}
?>

system

This entire folder gets copied to the machine being imaged. Some of the paths here may look a bit odd it is just to make their final copying simpler. 

system/
 cyclone.sh

cyclone.sh the main script that runs on the client to control the whole imaging cycle See here for full details of the system (Client Side Deployment System)

BASE

This is mostly a finisher for Win XP, it sorts out driver conflicts using the DP ini files which are part of the packs. This is the system as put together by driverpacks.net team :) 

devcon.exe   DP_Install_Tool.cmd  DPInst.xml    DPsFnshr.exe  DSPdsblr.exe  mute.exe     ROE.exe
DevPath.exe  DPINST.exe           DPs_BASE.exe  DPsFnshr.ini  makePNF.exe   pmtimer.exe  setup.exe

mbr

Contains Win XP/Win 7 boot records. MBR code is less important than you may think - win7 and server 2008 use came code. Syslinux MBR could replace all of them.

<MBR(partition table)><partition1>...<partition4> 

win7-mbr-446  winxp-mbr-446 server2008-mbr-446

ps

PostScript label source template for label printing ( ps/uol-label.ps) . This gets modified on the fly to create a machine label that is printed out, containing things like machine name, MAC, serial number etc. It also encodes this data into a 3D QR barcode purely in Postscript, thanks to super clever Terry Burton for this bit! The idea is if you wish to quickly audit a room of machines you can go around with a scanner and zap them all quickly to get a completely accurate data set.

This is the small snippet of the PostScript code, the HHH,SSS,MMM etc are replaced with various information gathered from the machine/dmidecode data. 

GS
1.1 -1.1 scale
148 -65 M (HHHHHHHHHH,SSSSSSSSSSSSSSSS,MMMMMMMMMMMMMMMMM,DDDDDDDDDD) (version=3 eclevel=L) /qrcode /uk.co.terryburton.bwipp findresource exec
%150 -65 M (HH,SSSS,MMMMMM,DDD) (version=3 eclevel=L) /qrcode /uk.co.terryburton.bwipp findresource exec
GR

reg

Windows registry files to inject after imaging. Currently only used for winXP. 

cddb.reg  winxp-RunOnceEx.reg
  • RunOnce, adds key to RunOnceEx to begin the Windows portion of the setup (join to domain, install software etc.)
  • CriticalDeviceDatabase (CDDB), injects MassStorage drivers into the CDDB and service sections of the registry, allowing it to boot without the dreaded 0x7b stop error! This also lets AHCI mode work in XP without problem.

sysprep

WinXP sysprep 

$ tree /cyclone/system/sysprep
/cyclone/system/sysprep
├── i386
│   └── $OEM$
│       └── CMDLINES.TXT
└── sysprep.inf

sysprep.inf

  • the basic WinXP sysprep file, modified on the fly during imaging to alter HAL etc.
  • HAL only gets 'try and update to...' (UpdateHAL=) from default (standard HAL) if it needs to be.
;richud.com
[Unattended]
    OemSkipEula=Yes
    InstallFilesPath=C:\sysprep\i386
    ;UpdateHAL=
    UpdateInstalledDrivers = Yes
    DriverSigningPolicy = Ignore
    NonDriverSigningPolicy = Ignore
    OemPreinstall=Yes

[GuiUnattended]
    AdminPassword=*
    EncryptedAdminPassword=NO
    AutoLogon=Yes
    AutoLogonCount=1
    OEMSkipRegional=1
    TimeZone=85
    OemSkipWelcome=1

[UserData]
    ProductKey=xxxxx-xxxxx-xxxxx-xxxxx-xxxxx
    FullName="*"
    OrgName="*"
    ComputerName=*

[Display]
    AutoConfirm=1

[RegionalSettings]
    Language=00000809

[Identification]
    JoinWorkgroup=WORKGROUP

[Networking]
    InstallDefaultComponents=Yes

cmdlines.txt

  • anything needing to run as XP boots prior to most other things, currently just for PnP cleanup
  • remember there is no command shell running, so any .batch script etc needs cmd /c running before it.
[COMMANDS]
"C:\sysprep\sysprep -clean"

win7

Win7 post deployment driver and software scripts

  • Installs software that changes a lot more than the image (Flash etc.)
$ tree /cyclone/system/win7
/cyclone/system/win7
├── cyclone-win7.bat
├── dpinst64.exe
├── DPINST.exe
├── dpinst.xml
└── software
    ├── AdbeRdr1012_en_US.exe
    ├── CertMgr.Exe
    ├── install_flash_player_ax_64bit.exe
    ├── lerootca.der
    ├── savw_97_sa_sfx.exe
    ├── sophos-97-silent.exe
    ├── sw_lic_full_installer.exe
    └── win7-eduroam.xml

cyclone-win7.bat

@echo off
TITLE CYCLONE by RMDC


IF /i "%PROCESSOR_ARCHITECTURE%" == "AMD64" (
	rem Install Drivers
	%~dp0dpinst64.exe /path %SystemRoot%\inf\x64
	rem Rearm and activate Office 2010
	"%PROGRAMFILES(x86)%\Common Files\microsoft shared\OfficeSoftwareProtectionPlatform\OSPPREARM.EXE" >nul
	"%SystemRoot%\system32\cscript.exe" "%PROGRAMFILES(x86)%\Microsoft Office\Office14\OSPP.VBS" /act >nul
)

IF /i "%PROCESSOR_ARCHITECTURE%" == "X86" (
	rem Install Drivers
	%~dp0dpinst.exe /path %SystemRoot%\inf\x32
	rem Rearm and activate Office 2010
	"%PROGRAMFILES%\Common Files\microsoft shared\OfficeSoftwareProtectionPlatform\OSPPREARM.EXE" >nul
	"%SystemRoot%\system32\cscript.exe" "%PROGRAMFILES%\Microsoft Office\Office14\OSPP.VBS" /act >nul
)


rem Update all accounts to not expire
rem NET ACCOUNTS /MAXPWAGE:UNLIMITED
wmic useraccount where "Name like '%%'" SET PasswordExpires=FALSE  2>nul >nul

rem Change passwords
net user localuser xxxxxxxxx >nul
net user localadmin1 xxxxxxxxx >nul

rem Install Adobe Reader
rem http://ardownload.adobe.com/pub/adobe/reader/win/10.x/10.1.1/en_US/AdbeRdr1011_en_US.exe
%~dp0software\AdbeRdr1012_en_US.exe /sAll /rs /l /msi /qn /norestart ALLUSERS=1 EULA_ACCEPT=YES SUPPRESS_APP_LAUNCH=YES DISABLE_ARM_SERVICE_INSTALL=1

rem Install Flash
%~dp0software\install_flash_player_ax_64bit.exe -install

rem Install Shockwave
rem http://www.adobe.com/go/sw_full_exe_installer
%~dp0software\sw_lic_full_installer.exe /S

rem Setup Eduroam
%~dp0software\certmgr.exe /add %~dp0software\lerootca.der /s /r localMachine Root
netsh wlan add profile filename=%~dp0software\win7-eduroam.xml user=all

rem Sophos Install
%~dp0software\savw_97_sa_sfx.exe /S
start /wait %~dp0software\sophos-97-silent.exe


timeout /t 5
rmdir /q /s c:\savw_97_sa

dpinst.xml

This will install all 'present' hardware. Windows uses its PnP installer to choose best match if mutliple DP's support same hardware. 

<?xml version="1.0"?>
<dpInst>
	<enableNotListedLanguages/>
	<suppressWizard/> 
	<quietInstall/> 
	<suppressEulaPage/>
	<scanHardware/>
	<legacyMode/>
	<search>
		<subDirectory>*</subDirectory>
	</search>
</dpInst>

Windows

Confusingly named, was left this way so it easily copied over from cyclone.sh keeping same path structure as what it overwrites.

Win7 sysprep file (unattend.xml) 

$ tree /cyclone/system/Windows
/cyclone/system/Windows
└── System32
    └── sysprep
        └── unattend.xml

unattend.xml

<ComputerName>*</ComputerName> modified on the fly by cyclone.sh post imaging, passwords or anything else could be too. 

<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend">

   <settings pass="oobeSystem">
	<component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <OOBE>
   		<HideEULAPage>true</HideEULAPage>
  		<SkipMachineOOBE>true</SkipMachineOOBE>
  		<SkipUserOOBE>true</SkipUserOOBE>
            </OOBE>

      <AutoLogon>
         <Password>
            <Value>xxxxxxxx</Value> 
            <PlainText>true</PlainText> 
         </Password>
         <Username>Administrator</Username> 
         <Enabled>true</Enabled> 
         <LogonCount>1</LogonCount> 
      </AutoLogon>

	<FirstLogonCommands>
                <SynchronousCommand wcm:action="add">
                    <Order>1</Order>
                    <Description>RMDC Post Installer</Description>
                    <CommandLine>cmd /C c:\win7\cyclone-win7.bat</CommandLine>
                </SynchronousCommand>
                <SynchronousCommand wcm:action="add">
                    <Order>2</Order>
                    <Description>Cleanup</Description>
                    <CommandLine>cmd /C rmdir /s /q c:\win7</CommandLine>
                </SynchronousCommand>
	</FirstLogonCommands>

        </component>
    </settings>

    <settings pass="specialize">
        <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
		<ComputerName>*</ComputerName>
        </component>
    </settings>

</unattend>

winxp

Windows XP post deployment files, replacement kernels.

$ ls /cyclone/system/winxp 

AdFind.exe   cyclone-winxp.bat  grep.exe      halacpi.dll  hal.dll       hwcheck.bat  hwcntrl.exe  NETUSER.EXE  ntkrnlmp.exe  sleep.exe
Au3Info.exe  devcon.exe         halaacpi.dll  halapic.dll  halmacpi.dll  hwcntrl.au3  netdom.exe   newsid.exe   sed.exe       tr.exe

cyclone-winxp.bat

@echo off
COLOR 18
mode con cols=60 lines=20
SETLOCAL ENABLEDELAYEDEXPANSION
SETLOCAL ENABLEEXTENSIONS
TITLE CYCLONE by RMDC

rem stopping System Restore service to speed things up
sc stop srservice >nul



rem imaging flow
rem fimg	hwchecks, no finisher , newsid, reboot, joindomain , clean
rem Domain	hwchecks, finisher , joindomain , clean
rem Standard	hwchecks, finisher , clean

rem using sysprep -pnp initially did or does STOP sysprep -clean working

rem hostn replaced by script
set u=joindomain
set p=%u%
set hostn=HOSTN
set runcmd=RUNCMD

rem Stripping any whitespaces
 set hostn=%hostn: =%




rem ######################
rem ## non forced image ##
rem ######################
if not [%runcmd%] == [fimg] (

 rem Starting HW Controller in continue mode
  start "RMDC" "%~dp0\hwcntrl.exe" continue

 rem Wait for any new hardware detection to start
  call "%~dp0\hwcheck.bat" wait

 rem Redetect PnP devices with devcon incase things missed due to dependant hardware 
 echo Re-scan for new devices
  rem start /wait "RMDC" RunDll32.exe Syssetup.dll,UpdatePnpDeviceDrivers
  rem altered 11 aug 2011 changed grep.exe -E "PCI|ACPI|USB|HDAUDIO to be ^ start line match as was mathicng USB in text on line below
  FOR /F "delims=" %%d IN ('%~dp0\devcon.exe status * ^| %~dp0\grep.exe -B2 -E "has a problem|PCI Device" ^| %~dp0\grep.exe -E "^PCI|^ACPI|^USB|^HDAUDIO" ^| %~dp0\sed.exe "s/\([^\]*\)\\[0-9].*/\1/g"') DO (
   echo Restarting device "%%d"
   "%~dp0\devcon.exe" restart "%%d"
  )
  
  rem altered 11 aug 2011 - needed to fix non restarting USB on DQ965
  start /wait "RMDC" RunDll32.exe Syssetup.dll,UpdatePnpDeviceDrivers

  rem slow machines need pause here as takes a few seconds for newdev.dll to get started 
  "%~dp0\sleep.exe" 10
 
 rem Wait for any new hardware detected to finish
  call "%~dp0\hwcheck.bat"

 rem DriverPack Finisher
  "%systemdrive%\DPsFnshr.exe"

 rem Wait for any new hardware detected to finish
  call "%~dp0\hwcheck.bat"

 rem Killing task
  taskkill /F /T /IM hwcntrl.exe 2>nul >nul

)

rem #######################
rem ## forced image fimg ##
rem #######################
if [%runcmd%] == [fimg] (
 if /i [%computername%]==[%hostn%] (
  rem newsid will have run if this is true
  goto joindomain
 )
 rem for domain and not sysprepped
 rem Starting HW Controller in continue mode
  start "RMDC" "%~dp0\hwcntrl.exe" continue
 rem Wait for any new hardware detection to start
  call "%~dp0\hwcheck.bat" wait
 rem Rerun on reboot, must be before newsid
 REG ADD "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnceEx\920" /V 1 /D "cmd /c \"%~dp0%~nx0\"" /f >nul
 echo Generating new SID and Renaming machine to %hostn%
 rem NewSID will cause AutoLogon to fail if;
 rem if AutoLogonCount has reached zero , running newsid and setting it back to 1 wont work (will if not using newsid)
 rem if AutoAdminlogon was set to 0 , running newsid and setting it to 1 wont work (it gets set back to 0 with reboot no matter what)
 rem changing the admin password here wont work either - the DefaultPassword gets erased if wasnt a password set previously
 REG ADD "HKCU\Software\Sysinternals\NewSID" /V EulaAccepted /t REG_DWORD /D 1 /f >nul
 "%~dp0\newsid.exe" /a /n %hostn%
 rem run newsid last as it breaks things like RunOnceEx also wont erase runonceex default as erased after key run
 goto restart
)

rem ######################
rem ## universal domain ##
rem ######################
if [%runcmd%] == [Domain] (
 goto joindomain
) 

rem ########################
rem ## universal standard ##
rem ########################
if [%runcmd%] == [Standard] (
 rem setting up generic local accounts
 net user localuser xxxxxxxxxx /ADD >nul
 "%~dp0\netuser.exe" "localuser" /pwnexp:y >nul
 net user localadmin1 xxxxxx%hostn% /add >nul
 "%~dp0\netuser.exe" "localadmin1" /pwnexp:y >nul
 net localgroup Administrators localadmin1 /add >nul
 goto cleanup
)


rem #################
rem ## join domain ##
rem #################
:joindomain
echo Checking for existance of %hostn% on AD
FOR /F "delims=" %%h in ('%~dp0\adfind.exe -c -u xxx\%u% -up %p% -h xxx.xx.ac.uk -b "DC=xxx,DC=xx,DC=ac,DC=uk" -f "(&(objectCategory=Computer)(name=%hostn%))" -dsq') do set adr=%%h
IF NOT [%adr%.] == [.] (
 echo Found machine %adr% now joining to AD
 "%~dp0\netdom.exe" join %hostn% /domain:xxx.xx.ac.uk /userd:%u% /passwordd:%p% >nul
 rem Fixing remote assistance
 sessmgr.exe -service
 rem Adding GPO key to make sure it syncs on logon
 REG ADD "HKLM\SOFTWARE\Policies\Microsoft\Windows NT\CurrentVersion\Winlogon" /V SyncForegroundPolicy /D 1 /f >nul
 rem remove Sophos entry from SMS so it will get reinstalled
 REG DELETE "HKLM\SOFTWARE\Microsoft\SMS\Mobile Client\Software Distribution\Execution History\System\UL10017F" /f
 rem Forcing group policy update
 gpupdate /force >nul
 goto cleanup
) ELSE (
 echo Machine not found on AD, assuming something went wrong
 echo Pressing enter now will clean up and reboot
 echo If you want to try again terminate the batch with cntrl C and rerun cyclone.bat when fixed
 pause
 goto cleanup
)


rem #############
rem ## cleanup ##
rem #############
rem added D cleanup 14 june 2010 also with Everyone perms
:cleanup
echo Cleaning Up....
taskkill /F /T /IM hwcntrl.exe 2>nul >nul
del /q %SystemDrive%\*.exe 2>nul >nul
del /q %SystemDrive%\*.dll 2>nul >nul
del /q %SystemDrive%\*.xls 2>nul >nul
del /q %SystemDrive%\*.txt 2>nul >nul
rem STAC audio does/didnt install properly and try and run out of driverpack installation location
net stop stacsv 2>nul >nul
sc delete stacsv 2>nul >nul
rd /q /s "%SystemDrive%\D" 2>nul >nul
echo Setting local admin password to dfW2g1@S0r
net user administrator dfW2g1@S0r >nul
rem Reset DevicePath key
REG ADD "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion" /v DevicePath /t REG_EXPAND_SZ /d %%SystemRoot%%\Inf; /f >nul
rem Remove AutoAdminLogon
REG ADD "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /V AutoAdminLogon /D 0 /f >nul
rem Remove install folder when first user login, folder has Everyone permission so deletable
REG ADD "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnceEx\930" /V 1 /D "cmd /c rd /q /s \"%~dp0\"" /f >nul
REG ADD "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnceEx\930" /V 2 /D "cmd /c rd /q /s \"%SystemDrive%\D\"" /f >nul
goto restart


rem ################
rem ## restart pc ##
rem ################
:restart
echo Restarting System
shutdown -r -f -t 1
goto end


:end


hwcheck.bat

This is called several times during the post image process. It relies on status of newdev.dll to detect what is going on. Relies on external program tr.exe for output formatting and sleep.exe for sleep. Depending on how its called it can ;

Cause a delay until hardware starts to be installed
Cause a delay while hardware is being installed
@echo off
COLOR 18
SETLOCAL ENABLEDELAYEDEXPANSION
rem loop count 80 is not enough on Tecra M10
set l=80

:hwalive
IF [%1]==[wait] (
rem Waiting for rundll32.exe to start
echo Waiting for new hardware detection to start
 FOR /L %%t IN (1,1,!l!) DO (
 "%windir%\system32\tasklist.exe" /fi "MODULES eq newdev.dll" 2>nul | "%windir%\system32\find.exe" "rundll32.exe" 2>nul > nul
  IF !ERRORLEVEL! EQU 0 (
   color 13
   rem Detected rundll32.exe running looking at new hardware
   goto hwcheck
  ) ELSE (
   echo %%t | "%~dp0\tr.exe" -d "\n"
   rem echo [%%t/!l!] Waiting 10s for hardware install to start .....
   "%~dp0\sleep.exe" 10
  )

 )
)

rem recode with one for loop if bored oneday

:hwcheck
rem Waiting on all PnP hardware to finish detecting
echo Waiting for new hardware detection to finish
FOR /L %%t IN (1,1,!l!) DO (
 "%windir%\system32\tasklist.exe" /fi "MODULES eq newdev.dll" 2>nul | "%windir%\system32\find.exe" "rundll32.exe" 2>nul > nul
 IF !ERRORLEVEL! NEQ 0 (
  COLOR 12
  rem Detected PnP finished
  goto hwend
 ) ELSE (
  echo %%t | "%~dp0\tr.exe" -d "\n"
  rem echo [%%t/!l!] Waiting 10s for hardware install to finish  .....
  "%~dp0\sleep.exe" 10
 )
)
:hwend

hwcntrl.au3

This AutoIT script(aka Hardware Controller script) is compiled to hwcntrl.exe. Has been around several years as you can see and occasionally tweaked if anything new found. It should catch everything you are likely to come across.

It serves to do several things;

  • supress dialog boxes
  • go through any wizards that may pop up
  • catch some wierdness various installs may throw up
; AutoIt Version:  3.0
; Author: richud.com
; hwcntrl Hardware Control
;modified from the newest one could find on hdd ghost backup dec 07
;18 march 09 - fixed CmdLine array issue  that allready existed
;18 march 09 - added 8104 option to look on interwebs
;7 jun 11 added widcomm fix, rmeoved sysprep bits
;3 aug 11 added stac fix for elitebook 2540p

Dim $vr = "hwcntrl - Hardware Control"
If WinExists($vr) Then Exit  ; It's already running
AutoItWinSetTitle($vr)

Dim $tit[10]
Dim $act[10]

	;catch anything that didnt get a driver installed 
	$tit[0]="Found New Hardware Wizard"
	;CANCEL
	;$act[0]="[ID:2]"
	;YES 8104 , NEXT (look on interweb)8104 , NO 8105
	$act[0]="[ID:8104]"

	;catch anything that didnt get a driver installed 
	$tit[1]="Found New Hardware Wizard"
	;CANCEL
	;$act[1]="[ID:2]"
	;NEXT
	$act[1]="[ID:12324]"

	$tit[2]="Found New Hardware Wizard"
	;FINISH
	$act[2]="[ID:12325]"
	
	;catch any non signed drivers that the dpsdsblr misses - soft inst STOP ID 5306 
	$tit[3]="Hardware Installation"
	$act[3]="[ID:5303]"

	$tit[4]="Software Installation"
	$act[4]="[ID:5303]"

	;GUI setup phase while KTD running - restart pc YES ID 6
	$tit[5]="System Settings Change"
	$act[5]="[ID:7]"
	
	;confirm not to overwrite new files with old - 407 no to all - 6 yes 7 no
	$tit[6]="Confirm File Replace"
	$act[6]="[ID:407]"

	;WIDCOMM bluetooth install dont run silently says admin privaledges
	$tit[7]="Bluetooth Software Installer Information"
	;OK
	$act[7]="[ID:2286]"
	$tit[8]="Bluetooth Software -"
	;OK
	$act[8]="[ID:2136]"

	;STAC error elitebook 2540p problem
	$tit[9]="STacSV.exe - Application Error"
	$act[9]="[ID:1]"

If $CmdLine[0] > 0 Then 
 If $CmdLine[1] = "suppress" Then
	$act[0]="[ID:2]"
	$act[1]="[ID:2]"
 EndIf
EndIf

;MsgBox(0, $CmdLine[1], $act[0])

Func hwsupres()
	For $i = 0 To UBound($tit)-1
		If WinExists($tit[$i]) Then 
			If ControlCommand($tit[$i], "",$act[$i],"IsVisible","" ) And ControlCommand($tit[$i], "",$act[$i],"IsEnabled","" ) Then
				;MsgBox(0,$tit[$i],$act[$i])
				ControlClick($tit[$i],"",$act[$i])
			EndIf
		EndIf
	Next
EndFunc

Do
hwsupres()
sleep(1000)
Until 1=2
Retrieved from "https://www.richud.com/w/index.php?title=Cyclone_Server_Side_Deployment_Sytem&oldid=4397"