COBALT CONFIGURATION ENGINE
$Id: CCE.specification.txt,v 1.1 2000/03/16 00:23:27 jmayer Exp $

0. Terms

	For the purposes of this document, the words "handler," "event
	handler," and "actuator" are interchangeable.  


1. Overview:

	CCE is a combination object database and event dispatcher designed to
	manage configurations of Cobalt appliances.

	The CCE maintains the whole configuration state of the system.  All
	other system configuration files and such are generated from the 
	state of the CCE database by actuator routines, triggered by values
	being changed..


2. Objectives of this design:

	This architecture will provide standard mechanisms to access all
	facets of the system, from services to users.  This architecture
	also allows us to define common data (such as IP address) in a
	common place and validate it where needed.  Lastly, this
	architecture gives us a clean, single source for data, and allow
	the UI to work independently of the actual data format.

2.1. A clean API:

  	- All configuration information for an appliance is abstracted
	  out to a hierarchy of objects and properties.

	- To insulate the user completely from the actuator methods
	  that translate configuration settings into system changes.

2.2. Clustering & Migration:

  	- Logical configuration objects can be easily moved
	  from one platform to another.  Example: migrating a virtual
	  site from one RaQ to another.

	- Logical configuration objects can be easily copied from
	  one platform to another.  Example: synchronizing settings
	  across multiple failover systems.

	- Configuration objects may be exported and imported arbitrarily.
	  Examples: configuration backup and system diagnostics.

2.3. Developer Support (Extensibility):

	- 3PDs can extend the configuration information in the CCE
	  by adding new objects and properties.

	- 3PDs can create actuator methods for any properties in
	  the database -- their software is notified
	  when config properties they depend on are changed.  
	  This ensures that all the software components in the system
	  are 'synchronized' and operating off of a single master
	  configuration database.  

	  Therefore: Everything Interoperates.

	- New classes can extend old classes via inheritance.
	
	- CCE should be programming language neutral with respect to actuators.

2.4. Security:

    	- CCE is the final authenticator and arbitor of access permissions
	  to the database.  By centralizing this control to one place, we
	  create a choke-point for solving all security issues.

2.5. Safety:

	- strong checking on configuration parameters.

	- rejection of application of invalid parameter combinations.

	- the ability to roll back a failed batch of changes to a
	  safe, initial state.

2.6. Testability

  	- CCE is designed from day zero with a test harness and automatic
	  regression suite for every component in mind.

2.7. Accountability

	- CCE should keep perfect logs of every action it performs.

2.8 Value Add Cobalt

	- Service, support, and ease-of-use aside: the unstated objective
	  of CCE is to create value-add IP that argues for Cobalt product
	  over generic Linux, for technically enlightened customers.

2.9 Potential Expansion to a Distributed System

	- CCE should be designed in such a way that it is possible to
	  expand to more distributed-aware mechanisms.


3. Implementation

3.1 Implementation in a nutshell

	The CCE consists of three main parts:
	
	Object DB: maintains an object database that reflects the current
	configuration state of the system.

	Session Manager: listens on a local unix-domain socket for
	connections from client applications, which may read or write
	configuration settings.  Configuration writes pass through the
	atomic broadcast layer, which syncronizes changes on multiple
	hosts, and re-emerge in the session manager.

	Event Dispatcher: triggers an appopriate set of actuators when
	data is changed and changes are committed to the database.

	The session manager reads the Object DB to give data to its
	clients, and collects client change requests into atomic
	"transaction objects" which it hands to the Event Dispatcher.  The
	Event Dispatcher processes the transaction object, triggers the
	appropriate handlers, and when all handlers complete successfully,
	updates the Object DB.

Diagram : Backend internal architecture
----------------------------------------
                                                -------|
                                                |      |--->----------
      ---------------------      --------------------      | Handlers |
      | Session Manager   |<---->| Event Dispatcher |---    ----------
      | ------------------|      --------------------   |
      | | Atomic Broadcast|                             |
      | ------------------|----   -----------------     |
      ---------------------   |-->| DB Access Lib |<----
                                  -----------------
                                    |
                             _______V_____
                             \___________/
                             | Object    |
                             | Database  |
                             \___________/

3.2 The Inside-Out Object Model

	One way to think about the CCE is to think of it as an "inside out"
	version of the traditional object model.  Traditionally, objects
	encapsulate data, and define a set of accessor methods to manipulate
	that data.
	
	In the CCE model, the encapsulated data is the interface: the user
	modifies the object data as the interface for messaging the object.
	The object is then responsible for triggering whatever actions are
	necessary to resolve it's new data state.

	This may seem perverse at first, but makes sense in the context of
	building a configuration system engine, in which the primary
	manipulation method is the changing of configuration properties, and 
	the user is to be insulated from the actuators that translate the 
	property changes into system changes.


4. The CCE ODB

	The object database maintains three main pieces of information:
		1. all of the class definitions in the system, and
		2. all of the object instances and object properties that
		   comprise the system configuration state.
		3. the list of handlers currently registered with the event
		   dispatcher.
	
	Implementation: The ODB is layered on top of an RDBMS, such as postgres.
		   
4.1 CCE ODB class definitions

	A CCE class definition consists of: 
		- a class name 
		- a class version string (format: "1.0-Cobalt") 
		- a parent class (ISA)
		- a list of valid scalar property names and types.
		- a list of valid reference property names and classes.

	A derived class inherits all of the property names of its parent class.

	A scalar property is represented as a string value within the ODB.  It
	may have arbitrary meaning outside the DB.  Scalar properties can
	have some additional type information associated with them to 
	accelerate the rejection of invalid values.

	A reference property is a collection of references to other objects
	within the ODB.
	
4.2 CCE ODB instance data

	All objects within the ODB are identified by a unique numeric object id.

	The ODB keeps track of what objects have been created, what class each
	object belongs to, and all of the property values for that object.

4.3 CCE handler data

	The ODB keeps track of the list of actuators registered on each 
	class-property pair.  
	
	FIXME: not decided yet
	Implementation: This list should be flushed and regenerated whenever the
	actuator config files are changed (ie. a new package is installed).

4.4 Object Properties

	An object property is identified by a string name.  This name
	may consist of any alphabetic character or underscore ('_'), 
	followed by any combination of alphanumeric characters or
	underscores.  E.g.:  It must match the regex
	/[A-Za-z_][A-Za-z0-9_]*/.

	An object property's value can either be a single scalar value, or can
	be a collection of object references.

	A scalar is just an binary string.  There is no limit defined on the
	size of a scalar (except that of physical memory), but large scalars
	may impact the overall system performance.
	
	An object reference is just an object_id.

	It's natural to ask why all CCE doesn't support all four
	permutations:
	
		singleton scalars		singleton references
		arrays of scalars		arrays of references

	The answer is: we don't need them.  Keep It Simple.  All we really
	need for a configuration database is singleton scalars and
	aggregate references.  Everything else can be constructed from
	these two simple components.

	Some property names are reserved (see the section in Classes below).

4.5 Scalar property types

	Scalar properties have additional type information associated with them.
	This type information is only used to accelerate data validity
	checks (to error out when a property is SET, rather than when a
	change is COMMITed).
	
	CCE implements a number of built-in types, and also provides a hook
	for an external data validator if necessary.
	
	Internally defined types include:

		scalar		a string of arbitrary binary data
		word			a string of non-whitespace data
		alphanum		a string of alphanumeric data
		int			an integer
		float			a floating-point number
		boolean		a boolean value
		ipaddr		an ip address tuple
		hostname		[A-Za-z][A-Za-z0-9\-\.]*
		re:<regexp>		the regular expression 'regexp'
		list:class		a reference list of class 'class'

	External validators are defined as:
		extern:/usr/bin/myvalidator

	External validators are passed data on STDIN, and exit with the value 0
	for success or non-zero for failure.

4.6 Reserved Property Names

	The following property names are reserved for internal use by the
	CCE database implementation.  These are properties in the sense
	that actuators can be attached to them, and some may have
	meaningful data when read, but none may be written to.
	
		_NEW			whenever a new instance is created
		_DESTROY		whenever an instance is destroyed
		_CLASS		the name of the object's class
		_CLASSVER		the current version of the object's class
		_PARENTS		the list of objects referencing this object
		_CHILDREN		the list of objects referenced by this object
		_DEFER		(see Security.Model.txt for details)
		_READ_ALLOW		(see Security.Model.txt for details)
		_READ_DENY		(see Security.Model.txt for details)
		_WRITE_ALLOW	(see Security.Model.txt for details)
		_WRITE_DENY		(see Security.Model.txt for details)
		_CHMOD_ALLOW	(see Security.Model.txt for details)
		_CHMOD_DENY		(see Security.Model.txt for details)
	

4.7 Garbage Collection

	All objects must at some point connect to object 0 (root).  Otherwise
	they can get garbage collected.  Objects with a ref count of zero get 
	garbage collected during the first GC run after they are unlinked.
	Also collected by the GC are old transaction info/warning messages
	(see CSCP.spec.txt, COMMIT command).
	
	The actual interval for GC runs is undefined, but is guaranteed to
	be at least once per day.  This interval should not be relied upon 
	by clients.
	

5. The CCE Event Dispatcher

	The ED waits for a transaction object from the SM, executes the
	handlers triggered by the changes specified by the transaction
	object, and then (if successful) updates the ODB.

5.1 The Transaction Object

	The transaction object is just a collection of commands that
	change the state of the system (NEW, DESTROY, SET, LISTADD, 
	and LISTRM).
	
	The transaction object provides a facility to query the ODB, filtered
	through the changes presupposed by the transaction object.  So: if you
	ask the ODB what ObjX.PropY is, you'll get it's current value; if you
	ask the transaction object, you'll get the value it's about to change 
	to.

	FIXME: details of transaction object and SM->ED interface

5.2 Transaction processing algorithm

	When processing a transaction, the following algorithm is used:
	
	1. A list of handlers is constructed based on the changes in the
	   transaction object.  (changes where new value = old value are
	   eliminated).  The list of handlers is sorted and uniquified.
	   
	2. The next handler is shifted off the list of handlers and
	   executed.  The handler is added to the list of executed
	   handlers.  If the handler times out, it is assumed to have failed.

	3. if the handler returns ...
		failure: we undo the whole transaction as defined below.
		reschedule: the handler is added to the end of the handler list
			for later execution.
		success: we proceed.

	4. if the handler made any additional changes, those changes are
	   merged into the transaction object (changes where new value =
	   old value are eliminated).  Handlers triggered by the additional
	   set of changes are appended onto the end of the handlers list.
	   The list of handlers is uniquified.

	5. check for deadlock.  If deadlock is detected, the whole transaction
	   is aborted.  We undo the transaction as defined below.
		
	6. if the handler list is not empty, return to step 2 to execute the
	   next handler in the list.
	
	If the transaction fails, the whole transaction is rolled back in
	the following manner:
	
	1. The list of changes in current transaction object is flushed.
	   Objects that had been created are destroyed.
	
	2. we shift a handler off of the list of executed handlers and
	   re-execute it.
	
	3. if the handler returns ...
			failure: warn and ignore
			reschedule: handler is shuffled to the end of the list
			success: proceed
	
	4. if the handler made any additional changes, those changes are
	   merged into the transaction object and necessary additional
	   handlers are appended to the handler list.

	5. check for deadlock.  If deadlock is detected, report a catastrophic
	   failure to the user and give up.
	
	6. while the handler list is not empty, return to step 2.

5.3 Event Dispatcher - Actuator communication

	When the ED execs the actuator, the actuator's filehandles are set
	up such that:
		stdin	reads data from the ED
		stdout	writes data to the ED
		stderr	writes data to the ED's logging mechanism.
	
	The actuator then communicates with the ED using an modified version 
	of the normal protocol that session manager employs.  This protocol 
	is specified in the protocol section.

5.4 Specification of Actuators

	Classes are defined via the CSCP protocol (see CSCP.spec.txt).
	
	FIXME: not defined yet
	Actuators, however, are defined using a set of configuration
	files.  See 'Actuator Specification File' format for info.


6. Session Manager

	The session manager is the socket interface side of CCE.  It accepts
	connections, runs the CCE protocol on them, assembles transaction
	objects and queues them up for the Event Dispatcher.
	
	The Session Manager is a state machine.  When a client connects,
	the connection is in the Authentication state, and uses the CCE
	authentication protocol to authenticate.  When authentication
	completes, the connection switches to the read state, and the
	CCE command protocol can be used to read data.  If a client then
	wants to make system changes, they can begin a transaction, and
	switch to the transaction state.
	
	The CCE protocol and states are defined in the CCE protocol section.

	FIXME: ABL


7. The CCE Protocol

	The CCE protocol - Cobalt System Configuration Protocol (CSCP) 
	is derived from the Cobalt Remote Administration Protocol (CRAP) 
	that Management Console uses to speak with its agents.  It is a
	socket based protocol for performing manipulations and queries on an
	object database.

	The gist of the protocol is thus: the client issues commands
	delimited by the newline character.  The server responds with a
	numeric code, followed by textual information.  

	Typically, a CCE connection will pass through several states,
	each with it's own subset of the global set of commands:

		1. Identification state
		2. Authentication state
		3. Read state
		4. Transaction state
		5. Status state

	Additionally, every event handler (actuator) will enter and remain
	in the actuator state.

7.1. The CCE protocol message format

	Every message issued by the server consists of a message, terminated 
	by a newline ('\n') character.  All CCE messages begin with a four 
	digit message-id, a sigle space character (' '), followed by some 
	amount of space-delimited text.
	
	CCE-transmission ::= CCE-message NL
	CCE-message ::= code SP message-text NL
	
	where 'code' is a 4-digit number that uniquely specifies the
	type of message.
	
	In general, the following convention is used for message codes:
		0000 - 0999: system initiated messages
		1000 - 1999: informational : operation still running.
		2000 - 2999: success : operation completed successfully.
		3000 - 3999: warning : operation still running.
		4000 - 4999: failure : operation completed unsuccessfully.

	Below is a more comprehensive breakdown of message-id allocation.

	0000-0999: system initiated messages
	------------------------------------
	0000		unused
	0001		Session Timeout
	0002		Shutting Down
	0001-0998	reserved for system messages
	0999		Engine on fire.


	1000-1999: informational responses
	----------------------------------
	1000-1099	reserved
	1100-1998	command messages (see command list)
	1999		reserved


	2000-2999: successful completion messages
	-----------------------------------------
	2000		CCE announce message
	2001		authentication success
	2002-2009	reserved
	2010-2099	reserved for general success messages
	2100-2998	command specific success messages (see command list)
	2999		server asked to close session


	3000-3999: warning responses
	----------------------------
	3000-3099	reserved
	3100-3998	command specific warnings (see command list)
	3999		reserved


	4000-4999: error completion messages
	------------------------------------
	4000		reserved
	4001		authentication failure
	4001-4009	reserved
	4010-4099	reserved for general error messages
	4100-4998	command specific  errors (see command list)
	4999		emergency - server may not recover


	Command specific numbering can be found in the CSCP protocol
	specification.

	When the user enters a command, the user can expect any number of
	1000 or 3000 series messages, terminated by a single 2000 or 4000
	series message.  

	The CCE only sends data in response to a user command. The only
	exception is 0??? messages sent in event of a failure or timeout
	after which the server must disconnect from the client.

7.2 The CSCP Protocol
	
	see CSCP.spec.txt

7.3 Security Issues

	Unlike ManCon's CRAP, support for encrypted sessions is not built
	into the protocol.  This protocol is envisages to run over local
	unix-domain sockets, so packet sniffing is not a security
	concern.
	
	The thinking is that, if the CCE protocol is to be run over TCP/IP,
	a separate "helper daemon" can be created that will act as a relay
	between TCP/IP and the unix-domain socket.  This "helper daemon"
	could then implement whatever encryption protocol is convenient,
	without impacting the design of CCE itself.	
	
7.4 Object-level Access Control

	See Security.Model.txt


8. Database Implementation

	The proposal for version 1.0 is to use the filesystem as it's
	database.  This allows us to prototype and develop the system
	without the overhead of a ful database.  If performance is poor, we
	can move to more robust/speedy solutions later.

	Within the filesystem object tree, each object is represented as a
	directory.  Each property is a file within that directory, and each
	list is a sub-directory, containing symbolic links to other
	'objects'.  Details will be worked out during prototyping.


9. Object Model

	* UserPrivate
	* Entity
	    User
	    Group
	        Site
	            VSite
	* Service
	    MTAService
	    POPService
	    IMAPService
	    FTPService
	    DNSService
	    HTTPService

	FIXME: TBD



Appendix A.  Notes to keep in mind for development

	Modules need to have 'export', 'import', and 'upgrade' routines
	We must be consistent with locking an re-entrancy
		possibly lock each actuator as it executes, so it can't run
		concurrently?
	We must find a way to protect our intellectual property
	The CCE must be uber-smart about data integrity.  This includes
		row-locking, buffering (shadowing), etc
	For EXTEND - when a property is added, we must
		scan subclasses for namespace errors.
	For group expansion - we must watch for recursive definitions:
		Ga contains Gb, Gb contains Ga..
	

Appendix B.  Unanswered questions

	Should OIDs be indirect references?
	We haven't decided on trigger-registration design?


Appendix C.  Changelog

$Log: CCE.specification.txt,v $
Revision 1.1  2000/03/16 00:23:27  jmayer
Copying over the old project documentation.  Much of this still needs
to be brought up-to-date.

Revision 1.14  2000/02/07 23:06:29  thockin
Work done in NYC this week

Revision 1.13  2000/01/30 01:58:45  harris
/tmp/cvs13070baa

Revision 1.12  1999/12/14 21:21:33  thockin
Tue Dec 14 1999  Tim Hockin <thockin@cobalt.com>
 - Some reorg
 - started list implementation
 - need to pause and create test routines, before I continue
 - need to pause to finish docs

Revision 1.11  1999/12/13 19:16:12  thockin
Mon Dec 13 1999  Tim Hockin <thockin@cobalt.com>
 - Redid a lot of the underlying API within the exported API
 - Eliminated several MAX definitions
 - scalars are now allowed to be any size - no limit
 - a bit of code rework
 - now I need to move on and finish up the harder bits of the PAI
 - Jonathan and I need to merge some CSCP stuff with this - scalars and
   possibly more
 - First I need to get in gear on the Spec. - due this week

Revision 1.10  1999/12/09 20:39:50  thockin
Some doc updates
Checking in before revising the (currently hodgepodge) internal API for i
the ODB - a lot of reuse can be found.

Revision 1.9  1999/12/08 19:55:42  thockin
* Wed Dec 08 1999  Tim Hockin <thockin@cobalt.com>
  - Added the start of the FS ODB implementation
  - Some minor tweaks to various files
  - The odb.h and ODB.API doc are out of sync - need to be merged

Revision 1.8  1999/12/03 00:51:11  thockin
* Thu Dec 02, 1999  Tim Hockin <thockin@cobalt.com>
- CLASSACL command added
- cleanups
- reorg of a few commands
- CCE spec has a couple outstanding FIXME points, but it is 90% stable.  If
  you are working on this, please read and feedback ASAP.

Revision 1.7  1999/12/02 19:43:50  thockin
* Wed Dec 02, 1999  Tim Hockin <thockin@cobalt.com>
- Merged CSCP spec with jonathans overview of states.
- Added 'bogo-object' concept to spec
- lots of FIXMEs taken care of
- general cleanup, typos
- updated Security Model to reflect new discussions
- BG command allows transition from status -> read state
- more still needed wrt permissions
- almost done with CCE section.  merges/references from Sausalito.spec.txt
  still needed.

Revision 1.6  1999/12/01 01:29:22  thockin
Tue Nov 30, 1999  Tim Hockin <thockin@cobalt.com>
  - This checkin is LONG overdue
  - CSCP.spec.txt has been largely overhauled, clarified, and cleaned up.
  - CCE.spec.txt has been cleaned up a lot also.
  - Several key things remain:
  	-- Unanswered questions remain in CCE.spec
	-- Permissions/access control needs to be merged in
	-- a more complete BNF needs to be done
	-- a few FIXMEs remain
	-- CCE docs need to be merged into the main doc
	-- other branches need to be examined/discussed/merged, too
	-- time to kick it into high gear
	-- added a file containing issues that we are circumventing now,
	   but that will need to be addressed eventually, wrt locking

Revision 1.5  1999/11/19 10:27:03  jmayer
Checked in a formal proposal for the CCE Security Model.  Great jumping
jehosephat on a dive board!  This is hairy looking.

CCE.specification.txt: changed Revision to Id.  mmmYep.

Revision 1.4  1999/11/19 03:26:02  thockin
Changes, cleanups.  A few FIXMEs still await fixes.

Time to move on to the toplevel spec and try to get some integration.

Revision 1.3  1999/11/19 01:22:03  thockin
Whole hog overhaul.  Most details decided in recent meetings are in here,
much clarification has been made.  They are still not done, but much
closer.

Revision 1.2  1999/11/17 19:47:21  thockin
Overhaul:
   * Sausalito.spec is underway
   * CSCP.spec and ESCP.spec have been split from CCE.spec
   * lots of comments added to various files - grep for FIXME for my
     questions so far
   * I will continue on this more today, I just wanted to do a commit
     before I forgot

Revision 1.1  1999/11/13 03:27:48  jmayer
Checking in a draft specification for CCE.  huzzah!


