Db 0.01 1.1.0 delta

These are detailed notes on changes made between bioperl-db-release-0-01 and bioperl-release-1-1-0.

Bio::DB::Annotation

Support for new Root object and Annotation framework

Bio::DB::BioDB

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

Bio::DB::BioDatabasePSeqStream

Support for new Root object and Annotation framework

Bio::DB::BioSQL::AbstractQuery

Support for new Root object and Annotation framework

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::BaseAdaptor

Added quote() which is a shortcut to $dbh->quote() to escape characters offending SQL

Support for new Root object and Annotation framework

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::BioDatabaseAdaptor

Fixed remove_by_name bug

Support for new Root object and Annotation framework

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

More fixes to Registry system with bioperl-db, tested and working

Fixed silly bug in fetch by display_id

Registry in perl working for biosql and biofetch, moved to new_from_registry methods

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

First step at revamping the entire bioperl-db testsuite to make tests behave nicely, clean up after themselves, and refrain from using hard-coded PK assumptions.

Corrected a bug in BioDatabaseAdaptor (the first record was omitted from delete).

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Added simple caching option when fetching seq, default behavior is still lazy

Bio::DB::BioSQL::BioQuery

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::BioSeqDatabaseFetcher

Adding fetcher for BioSeqDatabases needed by registry, uses new method taking config hash

More fixes to Registry system with bioperl-db, tested and working

Registry in perl working for biosql and biofetch, moved to new_from_registry methods

Bio::DB::BioSQL::CommentAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

First step at revamping the entire bioperl-db testsuite to make tests behave nicely, clean up after themselves, and refrain from using hard-coded PK assumptions.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::DBAdaptor

Support for new Root object and Annotation framework

Added bulk parsing to load_seqdatabase, added some docs to script

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Fixed silly bug in fetch by display_id

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Added method to retrieve biodatabase.name entries from the DBAdaptor, and put additional restriction on Spieces Adaptor to restrict its returned species list to only those in the requested biodatabase.name

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Rewrote tests and fixed adaptors along tests up to remove.t (in alphabetical order). remove.t not complete yet. All tests up to remove.t pass without destroying/rebuilding the database. Problem with exception upon fuzzy location in fuzzy2.t and remove.t remains (tested with test data file versions without the fuzzy locations, and all worked fine).

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::DBLinkAdaptor

Support for new Root object and Annotation framework

Added bulk parsing to load_seqdatabase, added some docs to script

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Rewrote tests and fixed adaptors along tests up to remove.t (in alphabetical order). remove.t not complete yet. All tests up to remove.t pass without destroying/rebuilding the database. Problem with exception upon fuzzy location in fuzzy2.t and remove.t remains (tested with test data file versions without the fuzzy locations, and all worked fine).

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::DBXrefAdaptor

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Rewrote tests and fixed adaptors along tests up to remove.t (in alphabetical order). remove.t not complete yet. All tests up to remove.t pass without destroying/rebuilding the database. Problem with exception upon fuzzy location in fuzzy2.t and remove.t remains (tested with test data file versions without the fuzzy locations, and all worked fine).

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::OntologyTermAdaptor

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::PrimarySeqAdaptor

Support for new Root object and Annotation framework

Added bulk parsing to load_seqdatabase, added some docs to script

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Modifications to PrimarySeq to reduce trips to MySQL instance. Result 2% speed up wooo wooo :-)

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::QueryConstraint

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::ReferenceAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

Fixed a problem with representig NULL in the text import

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SeqAdaptor

Workaround for Bio::DB::Seq->seq_version problem

Added tiyn utility method to get descriptions by accession, useful for quick lookup all across ensembl

Use quote() on sequence description and keywords

Support for new Root object and Annotation framework

Added bulk parsing to load_seqdatabase, added some docs to script

Fixed a problem with representig NULL in the text import

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Adding power to the API. Retrieve species names, sequence lenghts and names in bulk or individually. Basically, just helping it to play nice with Genquire.

Realized that it makes more sense to return the accession rather than the display_id when sending back sequence names and lengths

Realized that it makes more sense to return the accession rather than the display_id when sending back sequence names and lengths

Was searching for display_ids that matched accessions, which isn't necessarily always going to work. My cock-up. fixed now.

Trying with no sequnece length getting to improve fasta dumping

Modifications to PrimarySeq to reduce trips to MySQL instance. Result 2% speed up wooo wooo :-)

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Removed mark-specific crud from load_seqdatabase.pl. Added additional argument to the get_all_available call of the SeqAdaptor such that you specify the biodatabase name in the request. This makes more sense, as you should only be working on one biodatabase at a time, presumably

Fixed code to work for loading fasta files, which result in simple Seq objects, not RichSeq objects. Have to set seq_version to zero instead of calling the method, and have to set accession to display_id rather than to unknown. Still needs some thought, first fix

Seq version set to null rather than zero

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Temporarily rolling back seq_versions to zero

First step at revamping the entire bioperl-db testsuite to make tests behave nicely, clean up after themselves, and refrain from using hard-coded PK assumptions.

Rewrote tests and fixed adaptors along tests up to remove.t (in alphabetical order). remove.t not complete yet. All tests up to remove.t pass without destroying/rebuilding the database. Problem with exception upon fuzzy location in fuzzy2.t and remove.t remains (tested with test data file versions without the fuzzy locations, and all worked fine).

Fixed more of the tests and adaptors along with them. Two tests remain. One new bug in Bio::Location::Fuzzy was revealed by the location adaptor fix.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Added simple caching option when fetching seq, default behavior is still lazy

Bio::DB::BioSQL::SeqFeatureAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Fixed key getting from biosql - chained up all_SeqFeatures

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Rewrote tests and fixed adaptors along tests up to remove.t (in alphabetical order). remove.t not complete yet. All tests up to remove.t pass without destroying/rebuilding the database. Problem with exception upon fuzzy location in fuzzy2.t and remove.t remains (tested with test data file versions without the fuzzy locations, and all worked fine).

Fixed more of the tests and adaptors along with them. Two tests remain. One new bug in Bio::Location::Fuzzy was revealed by the location adaptor fix.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SeqFeatureKeyAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Removed test warnings

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SeqFeatureQualifierAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SeqFeatureSourceAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Fixed more of the tests and adaptors along with them. Two tests remain. One new bug in Bio::Location::Fuzzy was revealed by the location adaptor fix.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SeqLocationAdaptor

Code for storing remote locations is not ready. The regexp producing values for remote_seqfeature_name table is commented out causing execute fail on empty values. As a temporary fix to allow file parsing to proceed, I commeted out the prepare and execute lines.

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Removed test warnings

Added check for qualifier value, avoids inting a string

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

First step at revamping the entire bioperl-db testsuite to make tests behave nicely, clean up after themselves, and refrain from using hard-coded PK assumptions.

Fixed more of the tests and adaptors along with them. Two tests remain. One new bug in Bio::Location::Fuzzy was revealed by the location adaptor fix.

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SpeciesAdaptor

Added bulk parsing to load_seqdatabase, added some docs to script

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Sql quoting code was well buggy now we *always* quote by using BaseAdaptor->quote this keeps us nice and robust and compatible

Adding power to the API. Retrieve species names, sequence lenghts and names in bulk or individually. Basically, just helping it to play nice with Genquire.

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Added method to retrieve biodatabase.name entries from the DBAdaptor, and put additional restriction on Spieces Adaptor to restrict its returned species list to only those in the requested biodatabase.name

Removed test warnings

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSQL::SqlQuery

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::BioSeqDatabase

Support for new Root object and Annotation framework

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Added simple caching option when fetching seq, default behavior is still lazy

Bio::DB::CacheServer::SeqDB

Support for new Root object and Annotation framework

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

Bio::DB::DBContextI

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

Bio::DB::PrimarySeq

Support for new Root object and Annotation framework

Modifications to PrimarySeq to reduce trips to MySQL instance. Result 2% speed up wooo wooo :-)

Added simple caching option when fetching seq, default behavior is still lazy

Bio::DB::Seq

Support for new Root object and Annotation framework

Fixed key getting from biosql - chained up all_SeqFeatures

Trying with no sequnece length getting to improve fasta dumping

Modifications to PrimarySeq to reduce trips to MySQL instance. Result 2% speed up wooo wooo :-)

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Added simple caching option when fetching seq, default behavior is still lazy

Makefile.PL

Added Makefile depend on DBI

Tiny change.

docs/HOWTO-MySQL.html

Added excellent html docs by Chris Dagdigian

scripts/bioentry2flat.pl

Added #! line, fixed option reading

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

scripts/bioenv_server.pl

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

scripts/caching_corba_server.pl

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

scripts/download_marker_data.pl

Updated URLs for marker download sites

scripts/load_genemap99_markers.pl

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

scripts/load_genethon_data.pl

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

scripts/load_marshfield_map.pl

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

scripts/load_seqdatabase.pl

Doc fixes, default sqlname is now identical to one used in bioentry2flat.pl script

Added bulk parsing to load_seqdatabase, added some docs to script

Fixed a problem with representig NULL in the text import

Updated the docs for load_seqdatabase. Bulk loading via tab-delimited flat files cuts loading time by a bit more than half. Tab-delimited files must be imported by hand using mysqlimport, but I am working on this, and also on making it possible to do incremental updates using this script, as well as from-scratch loading

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Moved a lot of functionality to BaseAdaptor added a lot of fixes to get things working with postgres a lot f these fixes are good for mysql too; there was a lot of problems ith referential integrity, trailing ids were being left all over the place. postgres complains abou these, so its a good way of finding these bugs

More fixes to Registry system with bioperl-db, tested and working

Ripped all of the bulk loading code out of bioperl-db because it doesn't play nicely with the new schema. It is unlikely that it will ever be re-written as it didnt give a particularly useful speedup versus the direct object writing. PLEASE TELL ME ASAP if you discover that the load_seqdatabase.pl script is not working as it used to - I have backup copies of everything I changed and can set things back to the way they were if I buggered them up.

Removed mark-specific crud from load_seqdatabase.pl. Added additional argument to the get_all_available call of the SeqAdaptor such that you specify the biodatabase name in the request. This makes more sense, as you should only be working on one biodatabase at a time, presumably

Populates new seqfeature_dbxref [you must have post-capetown branch of biosql-schema to have this table] other postgres fixes - uses insert_nopk to get rid of annoying warning when trying get_last_id() for tables with no primary key added some transactional awareness

Moved Bio::DB::SQL to Bio::DB::BioSQL. There may be more databases than BioSQL for persistent storage.

scripts/load_whitehead_markers.pl

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

scripts/validate_whitehead_markers.pl

Added factory BioDB.pm. Added database context interface. Moved DBAdaptor implementations to the root of their respective database directory trees.

sql/basicseqdb-mysql.sql

> > I have committed some code to bioperl-db > > * Fuzzy Locations are now handled, using the location_qualifier_value > table added to the biosql-schema during the hackathon. > > * Optimisations - all the features and locations for a > sequence entry are > now fetched in a few SQL calls rather than a number of calls > proportional > to the number of features. > > * Tidying - a lot of mysqlisms removed or pushed up to the BaseAdaptor > layer, to allow for easier postgres support. Added a few > generic ease of > use methods to BaseAdaptor to more clearly expose the logic in the > individual adaptor layer. > > * DBTestHarness now no longer uses the copy of the schema in the > bioperl-db directory. Instead it checks > ../biosql-schema/sql/biosqldb-mysql.sql - hmmm, this won't > necessarily fit > with the cvs re-organisation. How should we do this? An env > var seems a > bit nasty.

Removing schema in bioperl-db since it is now housed in the biosql-schema module in the biosql repository for all projects