It is described here the way mnoGoSearch 
processes different HTTP codes ( for SQL backend only). 
Pseudo-language is used here for explanation. 
I hope it is clean enough :-)


---------------------------------------------------------------
200 OK

	1) If -m command line argument ("force reindex") specified, GOTO 4.
	2) Comparing new MD5 with old one stored in database
	3) If MD5 are the same, next_index_time=Now()+Period, GOTO 7
	4) Parsing the document, creating word list
		and adding in "url" table all of the found HREFs
	5) Comparing created word list with old one stored in "dict" table
	6) Doing UPDATEs, DELETEs or INSERTs in table "dict" if something is different in word lists.
	7) Done

--------------------------------------------------------------
304 Not Modified 

	1) next_index_time=now()+Period
	2) Done

---------------------------------------------------------------
301 Moved Permanently
302 Moved Temporarily
303 See Other

	1) Deleting all words in table "dict" for current URL
	2) If "DeleteBad yes"  deleting this URL from "url" table
	3) If "DeleteBad no"   next_index_time=Now()+Period
	4) Adding an URL given in "Location: " header
	5) Done


----------------------------------------------------------------
300 Multiple Choices
305 Use Proxy (proxy redirect)
400 Bad Request
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not found
405 Method Not Allowed
406 Not Acceptable
407 Proxy Authentication Required
408 Request Timeout
409 Conflict
410 Gone
411 Length Required
412 Precondition Failed
413 Request Entity Too Large
414 Request-URI Too Long
415 Unsupported Media Type
500 Internal Server Error
501 Not Implemented
502 Bad Gateway
505 Protocol Version Not Supported

	1) Deleting all words in table "dict" for current URL
	2) If "DeleteBad yes"  deleting this URL from "url" table
	3) If "DeleteBad no"   next_index_time=Now()+Period
	4) Done

------------------------------------------------------------------
503 Service Unavailable
504 Gateway Timeout

	1) next_index_time=Now()+Period
	2) Done

------------------------------------------------------------------

