VITAL - second go at the first round of testing

(This post is by Peter and Tim)

The first round of repository software testing is focusing mainly on scalability. AANRO has 200,000 records and counting – so there's no point in looking at software that can't handle that. Once we have an idea of which software handles 200,000 records we will begin to look at how AANRO might ingest new records an maintain old ones.

We're all confident that Fedora itself can handle lots of records, it's whether they can be used that's up in the air.

First up was VTLS Vital, version 3.1. We reported on that last week.

Next up was an upgrade to VITAL 3.1.1.

What we did

We took a plain-vanilla installation of VITAL 3.1.1. with no configuration changes at all (ie all default indexes and settings) and pointed it at a repository containing first 12979, then 17020 records. This is a total of 29999 records/items.

Set up

Machine used: 8GB Centos LINUX with 2000MB RAM

VITAL installation was completed successfully, without any problems.

As we noted in the last post, VITAL 3.1.1 requires that Fedora object have a certain structure; it uses a thing called an alternate ID to tie together a datasteam (such as PDF file) and metadata about that stream, or its full text or a thumbnail image.

We will report on the process of adding ALT-IDs soon, but first we let VITAL go ahead and index the items. VITAL is the least fussy of the software we're looking at. It will happily work with any-old Fedora repository, while Fez and Mura both require some things to be set up in a particular way before they can work with items.

Ingestion

12979 items ingested commenced approximately 9:50 am

All 12979 items successfully ingested into Fedora at 10:40am

All 12979 items successfully index into VITAL at 11.20 am

Further 17020 items ingested at approximately 11.25 am

All 17020 items successfully ingested into Fedora at 12.25 pm

All 17020 items successfully indexed into VITAL at 1.25 pm

Performance

During 12979 items being ingested – Takes 1 minute to load a Show All page.

With 12979 items ingested – Takes 10 seconds to load Show All page.

During further 17020 items being ingested – Takes 1 minute to load Show All page.

With 29999 items ingested – Takes 10 seconds to load Show All page.

With 29999 items ingested – Search for “pig” took 2.5 seconds to load

With 29999 items ingested – Displaying a page from page range took 15 seconds to load.

Copyright 2007 The University of Southern Queensland

Content license: Creative Commons Attribution-ShareAlike 2.5 Australia.