Bots version 2.1.0
This version is the first version in one-and-a-half year.Bots 2.0.2 proved to be quite stable.
This version adds a lot of new functionality; quite some bugs have been fixed.
This new verson is very upward compatible with version 2.0.2; some might experience a problem because of an extra grammar check; that is easily solved, see below in details.
- Highlights:
- new communication protocols: SFTP (SSH), IMAP, FTP with implicit SSL (ftpis).
- simpler database connector.
- handle excel, zipped files.
- performance improvements for x12 (large grammars).
- in GUI: persistent filters, sort order and selections.
- works with django 1.3.
- ssl suport for bots-webserver.
- limit the time an in-channel is using.
- MIND: some (edifact) grammars might give errors!
- Some (eg older edifact) grammars can give errors.
- This is due to errors in grammars, that are now being detected by bots (grammarcheck).
- The records in a grammar are now checked for unique field-names: the same field name is not allowed in a record! (never was allowed, but was never checked)
- Typically error: GrammarError: Grammar "...usersys/grammars/edifact.ORDERSD96AUNEAN008", record "FII": field "C078.3192" appears twice. Field names should be unique within a record.
- culprit is the file D96Arecords (or similar), the FII segment has an error in it.
- Solution 1: adapt grammar manually; change FII segment:
- - ['C078.3192','C',35,'A'],
- - ['C078.3192','C',35,'A'],
- to:
- - ['C078.3192','C',35,'A'],
- - ['C078.3192#2','C',35,'A'],
- Solution 2: use plugin 'update_edifact_recorddefs.zip' (same directory as update-plugins. this plugin only contains edifact records for D93a and D96a).
- Deprecated:
- Deprecated means: supported in this version (2.1.*), but not in the next (bigger) release.
- database connector with SQLalchemy. There is a new database connector; you can use SQLalchemy in this new database connector.
- code conversion via file (in bots/usersys/codeconversions). Use codeconversion via ccode tabel: better, more flexible.
- django 1.1.*. use django > 1.2.0.
- library kid (for html output) is deprecated, use Genshi instead. Genshi is the followed to kid.
- communication via intercommit. If needed, a plugin can be provided.
- use of 'UserCommunicationClass' in user communication scripts. Use (sub)class with standard name, eg 'ftp' instead.
- bots works with python 2.5, 2.6 and 2.7. At the moment there is no plan not to support older python versions. Development is targeting python 2.7. This means that new features might not be available for older python versions.
- List of changes
- bots-engine:
- added: limit the time an in-channel is using
- added: check for double field names in a record
- added: SFTP protocol (SSH)
- added: imap4 protocol
- added: FTP with implicit SSL (ftpis)
- improve: database connector. Old database connector (with SQLalchemy) still works. This one much easier to use.
- added: 'raw' editype
- added: unzip incoming files (via preprocess)
- added: extract pfd (via preprocess)
- added: extract excel files (via preprocess)
- improve: performance improvement (grammar.py); especially for (big) x12 grammars.
- added: confirm message (CONTRL) for edifact
- fixed: bug in exception-handling: transform.py, codetconversion and rcodetconversion
- fixed: error in mime-parsing
- fixed: bug in parsing mailbag
- improve: check if partners are known from editypes x12, edifact, tradacoms
- fixed: enveloping for xml
- added: use MIME header for outgoing mail
- fixed: bug in fields with min.length for x12 ,edifact: min length is now checked
- added: QUERIES for CSV - nextmessageblock
- improve: allow csv-files that have no terminating CR/LF
- fixed: bug in lexer: escaping the quote character in csv was not OK
- added: option to not check 'to' email address for incoming mail
- improve: better errors for errors in fields in incoming messages
- improve: pop3 handling (for bad internet connections)
- improve: sub-classing in communication
- improve: ftp incommunication
- added: other exit code for 'Database is locked, but "maxruntime" has not been exceeded.'
- added: user scripting for archiving
- added: user exit to determine email address
- added: user exits for bots-engine
- added: utility function for date formatting/conversion using masks
- fixed: cleanup seems to leave records
- added: block error report if only process errors
- improve: while archiving, a lot of empty folders are created for routes that are used occasionally.
- improve: handle alt different in finding translation (always use alt value for lookups)
- fixed: errortext could get to long
- fixed small bug in lexer: escaping the quote character in csv not always OK.
- fixed a bug for syntax parameter 'forcequote': in source and doc this was numeric, in grammar.py a string... Should have no consequences.
- new: csv with quoting for alphanumeric, no quote for numeric.
- added field format for numerical left aligned (no preceding zeroes) for fixed format.
- added field format for numerical right aligned (preceding blancs) for fixed format.
- added field formats for numerical left aligned (no preceding zeroes) for fixed format.
- Bots-monitor:
- added: persistent filter, sort order and selection in configuration screens
- improve: fixes for usage with django 1.3
- improve: changed SysTask deleting of transactions etc
- fixed: unlocking a not-locked database gives error
- added: search for user defined code list
- improve: in select: from ands to date/time default 00:00:00 and 23:59:59. Reason: Refresh will show newly arrived data.
- added: UserAdministration: Added first name, last name and email to users screen.
- fixed: "Datefrom" on select forms defaulted to 1860 days (about 5 years ago)
- improve: notify user if bots-engine is already running
- improve: removed default action from the "Run" menu
- improve: speed for document number search
- added: adminlimit: number of lines displayed on one screen for configuration items
- fixed: better errors when settings.py is not OK
- fixed: usergroup functionality in user administration did not work
- improve: process errors in communication did not report channel
- added 'manual confirm' for confirmation screen.
- fixed bug in bots-monitor: show html results directly in browser (via 'download').
- Plugin reading and writing:
- improve: write plugin: more fine-grained control what to generate
- fixed: when reading plugin for translate or confirmrule leads to double entries in database
- fixed: error for reading codelists in plugin
- fixed: compress generated plugins
- improve: add date to the filename of a generated plugin
- Other changes:
- added: ssl suport for bots-webserver
- improved: calling bots-engine for gui/bots-webserver works better with different environement.
- fixed initalisation errors for (command line) grammarcheck and botsupdatedb
- fixed: better handling of import errors/dependencies
- improved: logging of bots-webserver; different log per day; log could get quite large.
- for output to html/template: it is possible to use 'Genshi' library instead of 'Kid'. Genshi is the successor to Kid, Kid is not developed any longer.. For now Kid is still working, will be depreciated.
- Changes in bots.ini:
- Bots.ini contains several new options.
- If you use your (old) bots.ini: no problem, reasonable defaults have been used.
- the new bots.ini is in bots/install; you can check out which new options are added.
- bots-engine: