Thalia Image-Posting Performance Statistics

Tested 13 June 2007 against the isda-thalia2/3/4 set with unknown backend code, Alfresco Enterprise 2.0.1, and Oracle 10g:

• 1 item: 1.8 seconds
• 10 items, serially: 1.7 seconds per, 17 seconds total
• 100 items, serially: 2.0 seconds per, 208 seconds total
• 100 items, ten threads posting ten items each: 5.3s/58s (load ~4 on 2)
• 100 items, 100 threads with one item each: 40.0s/57s (load ~14 on 2, 4 on 3.)
• 1000 items, 1 thread with 1000 items: continued 2s average.  (Test terminated early.)
• 1000 items, 10 threads with 100 items each: 6.1s/816s (load 8/2)
• 1000 items, 100 threads with 10 items each: 15.6s/192s (load 12/4)
• 1000 items, 10 threads with 100 items each: 6.1s/816s (load 8/2)
• 1000 items, 100 threads with 10 items each: 15.6s/192s (load 12/4.  Alfresco starts breaking at this point, returning "org.alfresco.webservice.util.WebServiceException: Error starting session."

Tested 19-20 July 2007 against the isda-thalia5/6/7 set with unknown (but current for the date) IME, Alfresco Enterprise 2.0.1 (build 54, schema 41), and MySQL:

• 1 item: 3.1s
• 10 items, serially: 3.1s/31s
• 100 items, serially: 2.7s/278s (load 1/0)
• 100 items, ten threads posting ten items each: 8.4s/87s (2/1)
• 100 items, 100 threads with one item each: 77.9s/94s (5/2)
• 1000 items, 1 thread with 1000 items: 2.7s/2810s (1/0)
• 1000 items, 10 threads with 100 items each: 9.6s/974s, 9.0s/912s (load 3/3, spikes to 5/7; run twice)
• 1000 items, 100 threads with 10 items each: 88.4s/921s (load 4/2, spikes to 5/3)
• 250 items, 250 threads with 10 items each: 172.2s/2348s (load 4/2, spikes to 5/3, one spike to 11/3; start to see IME tomcat tempfails here but no real breakage)

Tested 31 jul 2007 on f5-frontended thalia.mit.edu group (aka isda-thalia5/6/7/8.mit.edu):
(backend code: 2007/07/27, revision 781, built 2007/07/30 13:07:34, UI: A24:07-31-2007, alfresco: build 54, schema 41, version 2.0.1)

• 1: 3.1s
• 1x10: 2.7/28
• 1x100: 2.4/247
• 10x10: 5.9/64 (1.5/1.5)
• 100x1: 58.7/76 (3/5)
• 1x1000: 2.4/2394
• 10x100: 5.6/629 (5/3)
• 100x10: 61.4/638 (4/3)
• 250x10: 140.4/1829 (1/4, spikes to 70/8; temporary failure start here)

• 50x10, 50x10 (two clients talking to different IMEs); IME load around 2, alfresco load around 4, and subjective performance from either IME vastly superior to one with 100 single-IME clients.)

Tested 10 Dec 2007 against the isda-thalia2/11 set (aka thalia-test.mit.edu) with IME #1404, Alfresco Enterprise 2.1.1 (build 189, schema 77), and (I think?) MySQL:

• 1 item: 2.743s
• 10 items, serially: 2.530s/26s
• 100 items, serially: 2.157s/220s (load 1/0)
• 100 items, ten threads posting ten items each: 5.392s/58s (2/1)
• 100 items, 100 threads with one item each: 54.474s/69s (3/6)
• 1000 items, 1 thread with 1000 items: 2.2s/~2200s (1/0)
• 1000 items, 10 threads with 100 items each: 5.666s/583s (5/2)
• 1000 items, 100 threads with 10 items each: 69.180s/712s (load 2/3, spikes to 3/7)
• 2500 items, 250 threads with 10 items each: 143.231s/~1800s (2/2, spikes to 4/7 and one IME spike to 19 with failures; IME reports "SocketTimeoutException: Read timed out", which is presumably Alfresco falling offline, and 503 errors talking to the IME which is presumably overload; some connections were also accepted by the IME but just hung forever)
• brief manual multithreaded testing (testing to two IMEs using the same Alfresco store) showed no problems and results similar to those above

Notes on data collection: in all cases times represent the time to complete the item import call from first byte sent to last response byte received.  (For users on a slow network, the upload time may be nontrivial as well.)  For the faster tests results were averaged across several runs but were generally consistent.  Median times were quite close to average times.  Image sets were the same between runs and come from the HST image collection.  Larger images take longer, though not linearly so.