Enhancing performance for huge documents with large tables

Catagory:Large Documents Lynn Hales and Suzanne Napoleon describe two different approaches to enhance the performance when editing huge documents containing large tables. Last Updated: 2006-09-13

=Question by N.N.=

Hi Adepters,

I was wondering if anyone had any ideas/tips for improving the performance of Epic when working with large documents/tables?

Our books are divided up by Chapter, section and then each record in that section is an individual XML document. We use Content@ as our XML repository and we tried in the past to split the document up into entities but this was unpopular with our users because they preferred to see the whole record they were working on while editing and were also unable to manipulate the record the way they wanted. Basically, they didn't have much flexibility in deciding what order these entities would go in inside the record.

We also have some books that are just packed with tables, very large ones too. I would turn off automatic update of generated text as Ed suggests but our users like to insert numbered notes in our tables and link them to data in the table. In the past they have kicked up a stink because the numbers weren't in sync (because generated text was not updated automatically), they don't like the extra work of having to click one more button on the toolbar. Finally, thank you Lupe for suggestion but I also don't think my users will like the idea of switching the tables into markup mode. They complain enough already about how tables are displayed in Epic. This will just give them one more reason to moan.

=Lynn Hales answers=

Are you working with the large document (I'll get to tables in a minute) for editing or publication? If you are editing, there are several ways to improve the performance, probably the best is to try to modularize the document. Make chapters (or similar blocks) individual files. Only bring all the pieces together for publication (or a QA prior to publishing). You can use the DTD Fragment to do this. The markup resembles this:

&lt;!-- Fragment document type declaration subset: ArborText, Inc., 1988-2003, v.4002 <!DOCTYPE MILSTD PUBLIC "-//USA-DOD//DTD MIL-STD-962C NON-CONFORMING  19991109//EN" [ ]> -->

The Fragment has to be inside a comment as does the DOCTYPE declaration AND (repeat AND) so do any locally declared ENTITIES in the declaration subset. Epic will then interpret the document making believe any missing parent elements are present. The DTD fragment is an OASIS TR and not an Arbortext extension like the <?Pub CX?> processing instruction:

<?Pub CX milstd(frontbody(>

Though Epic will insert one that emulates the path you are currently in. By editing a smaller chunk of you document, Epic does not have to parse, assign OIDs, generate text, etc for the entire document and it will run faster.

Now on to Ed's "Spawns of Satan" tables. You could modularize your tables to be file entities as well and edit them with the DTD fragment as well. However, if you have an extremely large table, you may not be able to improve your performance and breaking a table down is not something for the faint hearted I would not try it.

=Suzanne Napoleon adds=

If you can set hidesuppressed=on, I think the following should help. You could code the screen FOSI to suppress a table when an ati:display attribute is set to no. You would need to provide a menu item, toolbar icon, and/or keymapping for authors to turn table display on and off. The screen FOSI would display a message about how to toggle table display. It's an extra click for the authors, but gentext update could still be set to full. (Standard disclaimer here.)