BETYdb - BETYdb 5.4.1

sprockets will now also parse custom folder

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 3 years ago

Added

configuration to sync forks of BETY using https://github.com/wei/pull
configuration to set root_url

Fixes

Fixes to the docker build to fit in kubernetes helm chart.
Added Wisconsin back to database dump workflow
Catch exception when load.bety fails and cleans up /tmp folder
Fix problem with sendmail causing error on new user sign up https://github.com/PecanProject/bety/issues/677

Changed

#705 : More clearly indicate method is optional in bulk upload
updated rails to 5.2.4.6
removed static route, it was only used to render 404.
allow n=1 in bulk upload https://github.com/PecanProject/bety/pull/704

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 3 years ago

BETYdb - BETYdb 5.3.0

Please read the steps below to upgrade your system. Not following these steps could break your instance of BETY.

Password encryption

The previous versions of BETY would ignore the environment variable SECRET_KEY_BASE and use the default value defined in config/defaults.yml. If you have set SECRET_KEY_BASE and upgrade BETY you will not be able to login anymore!. It is best to remove SECRET_KEY_BASE if set and default back to the default SECRET_KEY_BASE.

This release requires a migration.

Steps Needed for Upgrade (Non Docker)

Log in to the server.
cd to the Rails root directory.
Run git pull to get the latest version of the master branch.
Update any gems using bundle
Run the migration in the new release with
bundle exec rake db:migrate RAILS_ENV=production
Restart the Rails server with
touch tmp/restart.txt

Steps Needed for Upgrade (Docker)

Log in to the server
cd to the directory with the docker-compose file
update the version of bety (if you don't use latest)
stop BETY
docker-compose rm --stop bety
download latest version
docker-compose pull bety
run migration
docker-compose run --rm bety migrate
start bety
docker-compose up -d bety

PEcAn dump script

There is a change to the script/dump.bety.sh that will keep the creation/update times of the users in the database. This is needed for future work to speed up database synchronizations. If you use the script standalone, please copy latest script.

Complete list of changes

Fixes

#696 : actually now uses environment variable SECRET_KEY_BASE. If this was set, unset otherwise you can not login to BETY.
#551 : remove GUnload message from console on page change.
#672 : Added activemodel-serializers-xml Gem to restore functionality of "original" API XML endpoints.
#674 : Upgraded comma Gem to restore CSV file downloads.
#699 : when dumping users created/updated is set to date of dump, now keeps original dates.
use actual information in database for load and dump scripts.
update doi links http://dx.doi.org --> https://doi.org
fixed functions to explicitly add public. to function call and table reference. (this will require a migration)
upgrade rake to 13.0.1 (dependbot fix)
upgrade nokogiri to 1.10.8 (dependbot fix)

Added

script to add new users to bety
dockerfile to dump database in docker image for faster restore of database.
switched to github actions instead of Travis
cronjob on github actions to create pecan/db docker image nightly, will safe also with year-weak (i.e 2020-15)

Renewable Energy - Bioenergy - Ruby
Published by robkooper about 5 years ago

BETYdb - BETYdb 5.2.2

This is a minor release and does not require and database updates

Changed

updated zenodo.json file to have a successful build and doi generated at zenodo
updated loofah and rubyzip gem to fix security issues

Renewable Energy - Bioenergy - Ruby
Published by dlebauer over 5 years ago

BETYdb - BETYdb 5.2.1

Currently the tested and recommended version of Ruby to be used with BETY is 2.3. BETY compiles and passes tests with 2.4, 2.5 and 2.6 but have not been tested thoroughly with these versions.

Added

#643 : Moved reindex.bety.sh and vacuum.bety.sh scripts from PEcAn to BETY and provided access through entrypoint.

Changed

Documentation updates, now point to new documentation location
#652 : Better list of soil choices
Ruby 2.3.8 is now default ruby when using RVM.

Fixed

Updated GEM list to fix security fixes as recommended by dependbot on GitHub.
Restores method assignment in Bulk Uploads
Fixed travis build, testing with 2.2, 2.3, 2.4, 2.5 and 2.6 of Ruby.
Fixed errors when running BETY with 2.4, 2.5 and 2.6
#668 : show "not found" page for invalid citations.

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 5 years ago

BETYdb - BETYdb 5.2.0

This release does require a migration to be executed.

Ability to set the initialize URL in docker
#505 : Added a new attributes table. This table allows to store through polymorphism any additional information with any row in any table. The value stored is a json document.
#597 : Moved dump.bety.sh and load.bety.sh from PEcAn to BETY.

Steps Needed for Upgrade

Log in to the server.
cd to the Rails root directory.
Run git pull to get the latest version of the master branch.
Update any gems using bundle
Run the migration in the new release with
bundle exec rake db:migrate RAILS_ENV=production
Restart the Rails server with
touch tmp/restart.txt

Renewable Energy - Bioenergy - Ruby
Published by robkooper about 6 years ago

BETYdb - BETYdb 5.1.0

Fixes

#611 : could not close alerts
#585 : Pressing "Show" on the Covariates Edit page attempts an update
#621 : A fix for a critical vulnerability reported by Danny Rosseau at Carve Systems (www.carvesystems.com).
#596 : Crop Model Maps links are broken
#605 : Allow for keyless API access (using guestuser access)
#618 : Can not download results of search as csv

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 6 years ago

BETYdb - BETYdb 5.0.5

Fixes

#603 : Restores Rails route to make method completion work in Bulk Upload Wizard

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 6 years ago

BETYdb - BETYdb 5.0.4

Fixes

#600 : Error when starting BETY as docker container due to frozen variable.

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 6 years ago

BETYdb - BETYdb 5.0.3

Fixes

#593 : Replaces references to gitter with references to slack.
#598 : Fixes issue in docker where the entrypoint referenced a wrong URL to download the initial schema from.

There are no schema migrations in this release.

Renewable Energy - Bioenergy - Ruby
Published by robkooper over 6 years ago

BETYdb - BETYdb 5.0.2

Update of Sprockets Gem Version

Summary of Changes

Per GitHub recommendation, the version of the sprockets Gem listed in the Gemfile.lock file has been updated from version 3.7.1 to 3.7.2.

Steps Needed for Upgrade

If you upgraded to release BETYdb 5.0 or 5.0.1 after 19:50 UTC on August 9 (2:50 PM CDT on August 9) with a git pull, you probably already got these changes. (Run git log to be sure.)

If you have not yet upgraded to Rails 5, simply follow the instructions for release BETYdb 5.0.

If you upgraded to release BETYdb 5.0 or 5.0.1 before 19:50 UTC on August 9, proceed as follows:

Log in to the server.
cd to the Rails root directory.
Run git pull to get the latest version of the master branch.
Run bundle.
Restart the Rails server with
```
touch tmp/restart.txt
```

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 6 years ago

BETYdb - BETYdb 5.0.1

Fix for API fuzzy matching

Summary of Changes

Changes to the Rails API introduced in Rails 5.1 caused problems with API version 1 fuzzy matching queries. This fixes that problem.

Steps Needed for Upgrade

If you upgraded to release BETYdb 5.0 after 12:01 AM UTC on August 8 (7:01 PM CDT on August 7) with a git pull, you probably already got this fix. (Run git log to be sure.)

If you have not yet upgraded to Rails 5, simply follow the instructions for release BETYdb 5.0.

If you upgraded to release BETYdb 5.0 before 12:01 AM UTC on August 8, proceed as follows:

Log in to the server.
cd to the Rails root directory.
Run git pull to get the latest version of the master branch.
Restart the Rails server with
```
touch tmp/restart.txt
```
Check that an API query that uses fuzzy matching (e.g. curl "<your BETYdb home page URL>/api/v1/search?treatment=~genotype&key=<your API key>) works.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 6 years ago

BETYdb - BETYdb 5.0

Upgrade to Rails 5

Summary of Changes

This release includes a major Rails upgrade—from version 4 to version 5—as well
as a few bug fixes.

Bug fixes

Add experiments table to data menu in the BETYdb Rails app #488

Up until now, the only way to visit the experiments table was to type the
correct URL into the browser address box. This fix adds a link in the Data
menu.
Can't create or edit Experiments via the BETYdb Web app #581

The Rails 4 upgrade introduced a bug preventing users from creating or editing
experiments via the BETYdb Web app. This problem is fixed by this release.
Getting ForbiddenAttributesError during search on TERRA-REF instance #582

Certain searchs via the version 1 API failed with a ForbiddenAttributesError.
This has been fixed.

Steps Needed for Upgrade

Note: This release is a major upgrade, so a few more steps than usual are
required. There are, however, no database migrations in the release, so users
may upgrade at their leisure and still be able to do data synchronizations.

To begin with, be sure the current directory is the Rails root directory and
that you are logged in as the BETYdb app user (the user that owns the files
under the Rails root directory, and the user that the BETYdb Rails app runs as).
Then do the following:

Check that you are currently on the 4.21 release. You can do this by running
```
git describe --tags --exact-match
```
It should return betydb_4.21. It's possible that you are on the correct version but somehow didn't pull
down the tag name. In that case, you should get
```
fatal: no tag exactly matches 'c7f98a8ab58acf0b2f68fde825f60b0dce236ab8'`.
```
To fix this problem, run git fetch and then re-run git describe --tags --exact-match. If you are not on the correct version, upgrade to BETYdb 4.21
before doing this upgrade.
Run git pull to get the latest version of the master branch.
Upgrade Ruby, if needed.

Rails 5 requires Ruby language version 2.2.2 or newer. If you already have
Ruby 2.2.2 or later installed, skip to the next step. Otherwise, continue as
follows:

If you are using RVM:

Ensure you are using the right version of Ruby and the right Gem set. Running
```
rvm current
```
should return ruby-2.3.0@betydb_rails5. If not, make sure you have ruby-2.3.0 installed:
```
rvm list
```
If it is not listed, run
```
rvm install "ruby-2.3.0"
```
Now make the Gem set "betydb_rails5":
```
rvm gemset create "betydb_rails5"
```
Run the following three commands to leave and return to the BETYdb Rails root
directory and check the current version of Ruby and the the current gemset:
```
cd ..
cd -
rvm current
```
The last command should return "ruby-2.3.0@betydb_rails5", automatically
creating the required Gem set if it does not exist. Alternatively, you can just
do
```
rvm use "ruby-2.3.0@betydb_rails5" --create
```
Note: If you ran the bundler with the --deployment option, Gems will be
stored under the BETYdb Rails root directory at vendor/bundle. In this
case, the RVM Gem set does not much matter. You can check if bundle is
using the --deployment option by running:
```
bundle config path
```
If you aren't using RVM:

Make sure you have Ruby version 2.2.2 or later installed, and make sure it is
the default version used:
```
ruby --version
```
should return version 2.2.2 or newer.
Run the bundler to get all the needed Gems stored in the right place:
```
bundle
```
Upgrade Phusion Passenger

It is strongly recommended you upgrade Phusion Passenger to version 5.3.4. If
you currently run a version of Passenger downloaded through the bundler, you
can continue to do so. The bundler will automatically download version 5.3.4
when you run the bundle command. To build the Apache module, run
```
bundle exec passenger-install-apache2-module
```
Choose Ruby as the language to build for. After the build is complete, the
script will print out updated LoadModule, IfModule, PassengerRoot, and
PassengerDefaultRuby directives for using in your Apache configuration
files. Edit your configuration files, replacing the existing
Passenger-related directives with these.

For more extensive Phusion Passenger installation instructions, consult
https://www.phusionpassenger.com/library/walkthroughs/deploy/, especially if
you are not planning to use the copy of Phusion Passenger found in the Gem set
maintained by the bundler.
Whether or not you are using RVM, you may need to update the version of Ruby
specified in your Apache configuration file. Run
```
passenger-config about ruby-command
```
to find what value to use with the PassengerRuby directive in the Apache
configuration file. (If you are using the copy of Passenger downloaded by the
bundler, you may need to prepend bundle exec to the foregoing command.)

Edit the configuration file as needed.

(If your existing configuration files do not override the
PassengerDefaultRuby directive with a PassengerRuby directive (inside a
VirtualHost block, say), then you do not need to include this
PassengerRuby directive. In this case, the path to the Ruby interpreter
found using the passenger-config command should match the path in the
PassengerDefaultRuby directive.)
If you haven't already done so, set SECRET_KEY_BASE.

This step should have been done in the upgrade to Rails 4. Here are the
instructions if it remains to be done:

Run
```
bundle exec rake secret
```
to generate a suitable value for this variable. The value must be in the
environment of the application. One way to do this is to use a directive in
an Apache configuration file of the form
```
SetEnv SECRET_KEY_BASE <secret key>
```
where <secret key> is the value you generated. This should go in the block
of the Directory directive corresponding to the path where your BETYdb Rails
code is located.
Compile Rails Assets

BETYdb now uses the Rails Assets Pipeline. You need to compile these
assets, and to do so, you need a Javascript runtime. You may already have one
on your server. If not, you can install one. On CentOS, for example, use
this command:
```
sudo yum install nodejs
```
On Ubuntu, replace yum with apt-get.

To compile the assets, first determine whether you serve your BETYdb site at a
subURI. For example, if the URL of the BETYdb home page is something like
https://www.mycollege.edu/bety, then /bety is the subURI and you should
use the RAILS_RELATIVE_URL_ROOT variable with the compile task like this:
```
bundle exec rake assets:precompile RAILS_ENV=production RAILS_RELATIVE_URL_ROOT=/bety
```
On the other hand, if you serve BETYdb from the root URL of your site, at
https://www.mycollege.edu, say, then omit the RAILS_RELATIVE_URL_ROOT
setting:
```
bundle exec rake assets:precompile RAILS_ENV=production
```
Restart the Apache httpd server to enable the updated configuration.
Test your site to make sure it is working.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 6 years ago

BETYdb - BETYdb 4.21

Add Foreign-Key Constraints and Database Maintenance Scripts

Summary of Changes

This release includes a migration to add all missing foreign-key constraints and changes the update rule of several existing ones to ON UPDATE CASCADE. It includes some Ruby scripts to detect and repair database problems. A few bug fixes are included and new features are added to the Bulk Upload Wizard.

New features

The Bulk Upload Wizard now supports putting the time of day into the date column and supports a method column (see issue #545).
Several scripts for diagnosing and fixing problems with BETYdb databases have been added. See the file script/db_maintenance/README for a summary.

Bug fixes

The migration in this release adds all missing foreign-key constraints.
The Covariate Show page has been cleaned up (see issue #280).
The Editing Format page has been fixed so that editing of related variables works properly (see issue #481).
reCaptcha VI Shutdown #588

This bug, which prevented new users from signing up, has been fixed.

Steps Needed for Upgrade

Log in to the server.
cd to the Rails root directory.
Check that you are currently on the 4.20 release. You can do this by running

git describe --tags --exact-match

It should return betydb_4.20. It's possible that you are on the correct version but somehow didn't pull
down the tag name. In that case, you should get

fatal: no tag exactly matches 'a763a495a2a9bb0a5ca3192b29d6e4e41832a03c'`.

To fix this problem, run git fetch and then re-run git describe --tags --exact-match.
If you are not on the correct version, upgrade to BETYdb 4.20 before doing this upgrade.

Run git pull to get the latest version of the master branch.
Get the new recaptcha Gem with

bundle

Run the migration in the new release with

bundle exec rake db:migrate RAILS_ENV=production

Restart the Rails server with

touch tmp/restart.txt

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.)

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 7 years ago

BETYdb - BETYdb 4.20 Upgrade to Rails 4.2

Upgrade to Rails 4.2; several bug fixes.

Summary of Changes

This release upgrades the version of Rails used from 3.2 to 4.2.

New features

Added Rake tasks for dumping and updating SQL comments using YAML files (issue #542 and pull request #543).
On the Advanced Search page, traits and yields are now searchable by method name (pull request #506)
When creating or editing Managements, possible management types now include "thining" and "other" (pull request #546).

Bug fixes

Issue #533 ("Trait/entity notes don't get populated through Bulk Upload Wizard")

If a Bulk Upload file has a "notes" column, those notes now show up in the newly-created traits or yields.
Issue #552 ("Travis fails trying to create postgis extension.")

Travis now runs RSpec successfully so builds no longer always fail.

Other Changes

Updated DBMS named in the README file from MySQL to PostgreSQL (long overdue).
Docker cleanup (pull requests #537 and #550).

Steps Needed for Upgrade

Log in to the server.
cd to the Rails root directory.
Run git pull to get the lastest version of the master branch.
Run bundle to upgrade several Gems (including Rails itself) and to add new Gems that are no longer part of Rails proper are that are needed to support legacy Rails behaviors until code is updated to no longer need this legacy support.
The file config/secrets.yml expects a variable called SECRET_KEY_BASE to be in the environment. To generate a suitable value for this variable, run rake secret.
Ensure this variable is in the environment seen by the HTTP server. If you run your BETYdb app under Apache HTTP server, one way to do this is to put a directive of the form

SetEnv SECRET_KEY_BASE some_secret_key

in the <Directory> section of the configuration file corresponding to the BETYdb instance you have deployed.
7. Restart the HTTP server, e.g. sudo apachectl restart.

[In lieu of steps 6 and 7, you could also edit config/secrets.yml directly, replacing

<%= ENV["SECRET_KEY_BASE"] %>

with the value you generated in step 5; then restart PhusionPassenger by running touch tmp/restart.txt. This is not the preferred way of doing things, however, since Git will then always show this file as modified.]

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.)

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 7 years ago

BETYdb - BETYdb 4.19

Publication of the beta API as version 1 (`/api/v1`); major speed-up of Bulk Upload wizard and several bug fixes.

Summary of Changes

Bug fixes

Issue #452 ("Unhelpful 'Bulk Upload' error message")
Issue #523 ("Can't delete variables via the BETYdb Rails app")
Issue #525 ("Pagination issue in BETY web interface when searching species") When editing a PFT and revising the species list, it is now possible to search for species and page correctly through the resulting list of search results.
Trying to visit an API endpoint of the form /api/v1/search/19 used to result in a cryptic error. Now it takes one to the /api/docs page (as do other invalid API routes) on which this is no longer listed as a valid route.

Other Changes

Published beta API as "v1" (version 1). Note that the beta API URLs are still valid.
Data access level 2 is now called "Internal & Collaborators" instead of "Internal EBI & Collaborators".
Major speed-up of the validation stage of the Bulk Upload wizard. This now should run up to two-thirds faster.

The remaining changes are probably of most interest to developers:

Migrated to using YARD (see http://yardoc.org) as the default application documentation generator, replacing RDoc. Running rake doc:app now runs YARD.
Simplified and better documented the XML schema used to validate XML files used in uploading new traits via the API.
Better documentation of other code, especially code implementing the Bulk Upload wizard.
Added extraction of the trait-upload XML schema documentation to the doc:app Rake task.

Steps Needed for Upgrade

Gem Installation

Administrators need to run the bundler to install the YARD gem.

Run bundle install from the Rails root directory.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command

 bundle exec rspec

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 7 years ago

BETYdb - BETYdb 4.18

Implements API query of sites by geographical coordinates; fixes constraints and sequence numbering of `sites_cultivars` table

Changes Pertinent to PEcAn Users

Administrators will need to do a database migration.

Summary of Changes

New Features

Querying sites by geographical coordinates

It is now possibly to query sites by geographical coordinates. For example, a URL of the form [hostname]/api/beta/sites?containing=40.064,-88.195 will return the site or sites containing the point at latitude 40.064 and longitude -88.195.

Bug fixes

Issue #521 (Two bugs in sites_cultivars)

The migration SitesCultivarsUniqueness guarantees that site_id is unique in the sites_cultivars table. This means sites referenced by that table uniquely determine a cultivar.

Steps Needed for Upgrade

Database Changes

Administrators will need to do a database migration.

The database version for this release is 20170712171513.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command

 bundle exec rspec

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 8 years ago

BETYdb - BETYdb 4.17

Important bug fixes to Bulk Upload Wizard

Summary of Changes

Bug Fixes

Issue #495 ("Yields with no assigned creator (user_id) are not updatable.")

Yields that have no assigned creator are now updatable. Moreover, the updater is, in this case, provided the option of being assigned as creator.
Issue #459 ("Change bulk upload (via wizard or API) to allow for traits not in the trait_covariate_associations table")

The implementation of this feature was flawed—in fact it would report that uploads were successful when in fact they were not. This has been fixed.
Issue #520 ("Bulk Upload Wizard shouldn't create unnamed entities if there is only one trait per row of the CSV file")

It is useless to attach a nameless entity to a single trait. This provides no useful information and only clutters up the database. Therefore, the Bulk Upload Wizard will no longer automatically create such entities.

Steps Needed for Upgrade

Database Changes

There are no new database migrations in this release.

The database version for this release is 20170415183619.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (Test spec/features/treatment_integration_spec.rb:74 passes intermittently.) There is one pending test having to do with site-group editing permissions.

The tests can be run using the command

 bundle exec rspec

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 8 years ago

BETYdb - BETYdb 4.16

Adds experiments API endpoint and new table relating sites to cultivars

Changes Pertinent to PEcAn Users

Administrators will need to do a database migration.

Summary of Changes

New Features

Issue #475 Add sites_cultivars table

Entries in the sites_cultivars table are used to specify that a given site is used for growing and studying a given cultivar. Trigger functions in the database schema ensure that this relationship is enforced on all traits and yields; that is, if a given row in the traits (or yields) table specifies a given site, and if that site matches an entry in the sites_cultivars table, then the row must also specify the appropriate cultivar as determined by that entry.

For more details, see https://github.com/PecanProject/bety/files/972071/sites_cultivars.pdf.

[At some future time this may be changed so that a cultivar may depend on both the site and the date of the trait or yield. This way, a site may be associated with different cultivars and different points in time.]
New API endpoint for experiments relation

Information from the experiments table is now available from the API endpoint
```
[BETYdb root URL]/api/beta/experiments
```

Other Changes

There are some minor changes to the new variable and edit variable forms, including a newly-available variable type "reflectance index".

Steps Needed for Upgrade

Database Changes

Administrators will need to do a database migration.

The database version for this release is 20170415183619.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command

 bundle exec rspec

Complete details for running the RSpec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 8 years ago

BETYdb - BETYdb 4.15.1

Security update. See https://github.com/PecanProject/private/issues/8.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 8 years ago

BETYdb - BETYdb 4.15

Assorted new features and bug fixes, including data upload improvements

This is the first major release in three months.

Changes Pertinent to PEcAn Users

Administrators may need to do a database migration.

If the BETYdb 4.14 - hotfix release was deployed, no migrations are necessary. See "Database Changes" below.

Summary of Changes

New Features

Issue #466 Add support for an entity column in CSV files for Bulk Upload.

An "entity" column heading in bulk upload CSV files is now recognized. The value in the column becomes the name of the entity to be used for all traits in the row. If the name matches an existing entity, that entity is re-used; otherwise a new entity will be inserted.
Issue #471 Add "experiments" view, show, and edit pages.

The experiments table is now viewable and editable via the BETYdb Web interface.
The BETYdb Rails application is now aware of the association between experiments and sites.
In the beta API, using the "key" parameter is now optional.

This is a convenience for users trying out the API in a browser: they now merely need to be logged in and don't need to include "key=" in the query string. Those using the API programmatically need not use the "key" parameter either if they supply a username and password.
Issue #432 In API, 'associated ids' should be optional (off by default)

A new parameter "associations_mode" affords control over how much information about associated tables to display. Recognized values include count (show the number of rows in another table that are associated with this item), ids (show a list of the ids of those rows), and full_info (show complete information about each associated item).
Extensions to site customization options:
- Whether or not to show the Model I/O menu item can now be set in the application.yml file.
- What place-holder text to show in the home page's search box can now be set in the application.yml file.
- The citation/license/copyright and contact information included in CSV downloads now matches the (customizable) information displayed in the Web page footer (part of issue #482)
Issue #482 Errors in search --> download, output csv

In addition to site-appropriate citation information, downloaded CSV files now include a "raw_date" field in addition to the interpreted date previously provided. Although "raw_date" can easily be misinterpreted, it may prove useful for some use cases.
Issue #459 Change bulk upload (via wizard or API) to allow for traits not in the trait_covariate_associations table

🔴 This feature is not working properly! For now, be sure all variables used as column headings are listed in the trait_covariate_associations table!

Any variable name used as a heading in a CSV data file that does not conflict with any of the usual heading names and is not listed as a covariate in the trait_covariate_associations table will now be recognized as a trait variable name. It is now also an error for there to be a heading that matches a recognized covariate name unless there is a matching trait name in the heading.

Improved and additional RSpec testing of trait-insertion API and Bulk Upload wizard.

Bug Fixes

Issue #472 There is an HTML class that is alternately spelled "final_paginate_row" or "final_paginagte_row".

For the most part, this has had no user-viewable effects.
Pull Request #478 Typo in 401 message

It now says "authentication" instead of "authenication".
Issue #479 species vs specie in new API

Species (singular and plural) are now referred to as such in API results.
Issue #485 Bulk Upload doesn't handle blank entries in "notes" column.

Previously either the "notes" column had to have a non-blank value in each row or the "notes" column had to be omitted altogether (in which case each trait inserted would have a blank note). Now it is possible to have notes for some but not all traits in the file.

Other Changes

The changes from https://github.com/PecanProject/bety/releases/tag/betydb_4.14-hotfix have been merged into the master branch. These changes have to do with the experiments table and related database tables and objects.

Steps Needed for Upgrade

Database Changes

Administrators may need to do database migrations!

If BETYdb 4.14 - hotfix was deployed, no migrations are needed. If you are upgrading from the BETYdb 4.14, then there is one migration which needs to be run.

The database version for this release is 20170118205944.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command

 bundle exec rspec

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 8 years ago

BETYdb - BETYdb 4.14 - hotfix

Hotfix release: Changes to the experiments table and related join tables.

This release adds a migration to set the starting id values for the experiments table and related join tables. It also adds constraints on those join tables so that joins are unique.

This release is on a branch. To update your code, run

git fetch
git checkout betydb_4.14-hotfix

Administrators need to do a database migration.

If you have any rows in your experiments, experiments_sites, or experiments_treatments tables, it is recommended that you dump the data in them, truncate the tables to empty them, and then re-add the data after having run this migration.

To update the database, run

bundle exec rake db:migrate RAILS_ENV=production

The database version for this release is 20170118205944.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 8 years ago

BETYdb - BETYdb 4.14

Requested database changes; changes to the API

This release implements requested changes to the likelihoods table and fixes some problems with the API.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

The trait-insertion API now supports the use of a notes column in submitted CSV files.

Bug Fixes

Issue #432 In API, 'associated ids' should be removed or optional (off by default)

Now, instead of a listing of all of the ids of associated rows in the associated table, a simple count of the number of associated rows is returned. (The option to get the associated ids if desired may be added at a later time.)
Issue #456 Beta API response does not set charset setting in Content-Type header

This bug showed up when a JSON API query was run in a browser and the results contained an item having a non-ASCII character. The character wasn't decoded properly as UTF-8. This has been fixed.
Issue #461 Need "nicer" error handling when beta API data requests have a bad query string

Fixed: The response will return the error in the requested format (XML or JSON).
Issue #462 Need "unique_run_variable_input_combination" constraint to be removed

In response to a user request, the included migration removes this constraint.
Issue #463 schema change for likelihoods for more precision

In response to a user request, all four likelihood columns of type numeric(10, 0) will be changed to floating-point type double precision by the included migration. There doesn't seem to be any advantage to preferring a numeric type to a float type for these columns.
Issue #465 Some Rails tests depend on now-customizable portions of the Home page

Now that many portions of the home page are customizable, tests that depend on seeing a certain textual item to test whether one is on the home page have been changed to a more robust method of testing for this.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There is only one migration in this release. It affects only the likelihoods table, changing numeric types (which allowed only integral values) to floating-point types and removing a uniqueness constraint.

The database version for this release is 20161129192658.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command

 bundle exec rspec

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 8 years ago

BETYdb - BETYdb 4.13

Extensions to the Traits and Yields View, a New Experiments Table, and a few New Features and Bug Fixes

Improvements to the trait-creation API include support for a "method" column when using CSV format.

The TraitsAndYieldsView now includes cultivar, entity, and method information. Also, date information is now in human-readable form.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Issue #368 — Most aspects of the home page and the header and footer are now customizable. The file defaults.yml has settings that will maintain the existing look of the home page, but users may now customize most aspects of the home page by overriding these defaults.
Issue #438 — The traits and yields view now includes cultivar, entity, and method information. Also, date information is now in human-readable form. The layout of the advanced search results page has been modified to accommodate the additional information.
The trait-insertion API now supports the use of a method column in submitted CSV files.
An experiments table and related join tables have been added to the database.

Bug Fixes

Issue #430 — The API code used to return a hard-to-decipher error when lookup of a covariate variable named in the submitted trait data file failed. This has been corrected.
Issue #435 — The benchmarks_ensembles_score table now includes a score column that was inadvertently omitted in a previous release.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There are four migrations in this release. There is only one minor change to an existing table: a score column is added to benchmarks_ensembles_scores. All of the other changes are additions of tables, additions of user-defined functions, or modifications of views. It is expected, therefore, that the migration step should be relatively painless.

The database version for this release is 20161005181021.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 8 years ago

BETYdb - BETYdb 4.12

API Changes, Bug Fixes, and a few New Features

Improvements to the trait-creation API include support for an "entity" column when using CSV format, and ensuring trait treatments are consistent with the trait citation.

The TraitsAndYieldsView resource is now available at /api/beta/search without error.

Summary of Changes

Bug Fixes

Issue #411 ("can't add sites to site-group") This has been fixed and an RSpec regression test has been added.
Issue #439 ("Can't update a site with NULL geometry") This has been fixed and an RSpec regression test has been added. Users must now explicitly set all three geometry coordinates (or leave them all blank).
A bug in the formatting of the "From" header of API-related e-mail messages has been fixed.
Certain problems having to do with Rails' treatment of PostgreSQL domains have been fixed.

Improvements and New Features

API-related changes

As mentioned above, when using the trait-creation API, trait treatments must now be consistent with the citation associated with the trait. This also means that the name used for specifying a the treatment for a trait must only be unique within the set of treatments associated with the specified citation.
As mentioned above, an entity column in now supported in CSV files submitted to the API's trait-creation pathway.
Anonymous entities are no longer automatically created for each trait when an entity is not specified explicitly. (Per-trait anonymous entities add no useful information.)
All registered users, no matter what their permission levels, are now allowed to change their own API key.
API search results now include an "edit" URL with each item.
When creating new traits via the API, fractional seconds are now allowed in the local_datetime field.
Additional validity checks have been added for documents using local_datetime to ensure a site with a well-defined time zone has been unambiguously specified.

Other changes

The "notes" Rake task (this is mainly of interest to developers): Running rake notes (or bundle exec rake notes) searches the code for notes and to-do items. This functionality has been extended so that more "special words" are recognized: namely "deprecation", "deprecated", and "note" (in various capitalization styles). And spec and script have been added to the list of directories searched.
When running RSpec JavaScript-based tests in debug mode, the Chrome driver is now used instead of Firefox. (Version incompatibilities made using Firefox a pain to deal with.)
The "Advanced Search" page (at https://www.betydb.org/search) now shows the "Include Unchecked Records" checkbox more prominently.

Steps Needed for Upgrade

Simply pull the new code and restart your server. No database migrations are required and no new Gems need be installed.

(Additionally, if you want to use the Chrome driver for RSpec tests, you must download it and put it in your path.)

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 8 years ago

BETYdb - BETYdb 4.11

Addition of Tables for Benchmarking

The release contains a migration that adds nine new tables to the database, all having to do with benchmarking. In addition, there are a few small bug fixes.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

Bug Fixes

Issue #421 (Wrong text on Site Show pages)
Issue #429 (API documentation location)

Users are now redirected to a page showing all available API paths when they visit any unrecognized path under /api.

Revised API key e-mails

The e-mail message users receive when setting an API key now directs them to the web page where that API key may be viewed.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There is one migration in this release, 20160711231257_create_benchmarks_tables.rb. It adds nine new tables, all having to do with benchmarking.

The database version for this release is 20160711231257.

Gem Installation

No new Gems need be installed.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.10

New Support for Uploading Traits via API

The release extends the API for inserting new trait data by allowing it to be submitted in CSV format. In addition, a few bugs are fixed, including broken map pages.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Trait Insertion API Accepts CSV Files

Documentation of this feature is available at https://pecan.gitbooks.io/betydbdoc-dataentry/content/trait_insertion_api.html.

Bug Fixes

Issue #401 (Model Projection page is broken)
Issue #423 (RGeo gem not functioning correctly)

Although this was not a problem at the betydb.org website, some other installations would return an error when the user attempted to view a Site having a non-point geometry.

This release updates the version of the RGeo Gem specified by the Gemfile and Gemfile.lock files. At test has been added to ensure that the RGeo Gem is functioning correctly. The test may be run from the Rails root directory by issuing the command

bundle exec rspec spec/miscellaneous/rgeo_spec.rb

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There is one migration in this release, 20160617133217_add_id_to_table.rb. It adds id columns to the cultivars_pfts table and to the trait_covariate_associations table. The latter addition should eventually allow the database-synchronization script to handle the trait_covariate_associations table properly.

The database version for this release is 20160617133217.

Gem Installation

Administrators need to run the bundler to update the RGeo Gem.

Status of RSpec Tests

All non-pending tests pass when run in the default environment. (There is one pending test having to do with site-group editing permissions.) The tests can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are at https://pecan.gitbooks.io/betydb-documentation/content/automated_tests.html.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.9.1

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.9

Add Time Zone Column to Sites Table

This release replaces the local_time column with the full IANA-compliant time zone name. These names are "daylight-savings-time-agnositic" so that given this name and any UTC date and time, PostgreSQL can correctly compute the corresponding local time at the given site.

Previously the sites table had a local_time column that was just an offset. It was not often used (only 80 out of 5905 sites), had integer datatype (so that it can't accommodate half-hour or quarter-hour offsets), and gave no information about local daylight savings time rules.

Motivation

Historically, when trait information has been collected and entered into the BETYdb database, it has been unclear what time zone was used to represent the time (in particular, whether local time or UTC time was used). The BETYdb Web interface to the traits table is currently being redesigned to display and allow entry of date and time information in a human-friendly way that automatically takes the dateloc, timeloc, and site time zone into account. Adding complete time zone information to the sites table is the first step in this process.

Visible changes

For now, the only change visible to users of the BETYdb web application will be on the sites "show" pages: Instead of displaying the (numerical) "Local Time" field, these pages will instead display the time zone name. Note that sites that span multiple time zones will have NULL in the time_zone column, and the time zone field on the "show" page will be blank for these sites.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

After Git-updating your BETYdb working copy, simply run the one migration included in this release, which removes the (integral) local_time column from the sites table and replaces it with the (textual) time_zone column.

The new column will be populated on the EBI site; sync with that site to populate the column on your own site.

If you add sites of your own to your own database, you will have to set the time zone manually; the BETYdb web interface does not support editing site time zones. The time zone name should match one of the names returned by the query SELECT name FROM pg_timezone_names, but this is not enforced.

To see a list of IANA time zone names, visit https://en.wikipedia.org/wiki/List_of_tz_database_time_zones. You may also see a list in PostgreSQL by running the query SELECT * FROM pg_timezone_names. (The name column is what you are interested in; the other three columns will change in value with the time of year. In particular, the time zone abbreviations shown in the abbrev column are not daylight-savings-time agnostic.)

If you are adding a large number of sites, you may want to (at least temporarily) add a time zone database table which will allow you to populate the time_zone column using an SQL UPDATE statement that derives the time zone from the site geometry column. See the comment https://github.com/PecanProject/bety/issues/416#issuecomment-221073159 for more information.

The database version for this release is 20160523165531.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command

bundle exec rspec

Complete instructions for setting up the test database and running the RSpec tests are on the Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.8.1

Security Release

This release fixes important security problems that may allow any public user with a valid API key to access and use other user accounts.

Visiting the path /api/beta/users?key=<any valid API key> shows all user data. Similarly, any registered user could also get all stored information about a specific user by visiting the path /api/beta/users/nnn?key=<any valid API key>, where nnn is the id number of some user.

Maintainers of public-facing BETYdb distributions should update as soon as possible. This is especially important if any privileged users have API keys.

If you are concerned that private user data has already been accessed, you may wish to check the logs. Do grep "api/beta/users" log/production.log from the Rails root directory and if any results are returned, check that the IP address listed corresponds to a known user. You may wish to regenerate all API keys.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.8

Added new site-grouping features. Minor bug fixes and upgrades. Ruby 2.1 or greater is now required.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Grouping of Sites

This release of BETY now allows the user to create a grouping of sites. The user can make this grouping publicly accessible or make it private for the user only. The grouping can be used in the BETYdb web interface to quickly get to sites of interest, or in the PEcAn interface to show only those sites the user is interested in.

Upgrade to RSpec 3

The version of RSpec that we use for testing has been upgraded from version 2.14 to version 3.4. This shouldn't affect end users, but developers will now be able to use the latest version of RSpec when writing new tests.

Other Notable Changes

Ruby Versions and Travis

Although Ruby 2.1.5 is the Ruby version "officially" supported by BETYdb, up until recently, BETYdb would run with versions 1.9.3 and 2.0. This is no longer the case.

Since tests will now fail under both Ruby 1.9.3 and 2.0.0, these version have been removed from the set of versions run by Travis (our automated testing service). Travis instead now runs tests under the following Ruby versions: 2.1.5 (the "official" version), 2.1.8, 2.2.4, and 2.3.0.

Although users are strongly encouraged to upgrade Ruby to at least version 2.1, users who still want to run BETYdb under older versions may be able to do so for the time being by making certain changes to their code. If you are a user who wishes to do this, please contact BETYdb support.

Bug Fixes

GitHub issues #386 and #387

An issue arose with bulk uploads when a trait or covariate variable was used that had a maximum value set to +Infinity (GH issue #386). This was because the code would treat such values as being zero, so that no positive value was acceptable (GH issue #387). (A minimum value of -Infinity was also treated as zero, making negative values unacceptable as well.) This has been fixed.

Problems with Travis and Database Migrations

A problem with Travis (an automated test-running service) used to occur when a database migration was committed. If any test code depended on the migration, the test would fail until the file db/production_structure.sql was updated. Even worse, if a test fixture depended on the migration, the test setup would fail and no tests at all would run. Since db/production_structures.sql is updated only upon deployment of new code to www.betydb.org, this meant that Travis tests would fail from the time the migration was first committed to GitHub until the time of deployment.

Now Travis uses the (new) file test_structure.sql. This file should be regenerated and the update should be added to the GitHub repository each time a new migration is added.

Steps Needed for Upgrade

Database Changes (if applicable)

Administrators need to do database migrations!

There is one migration, which adds the sitegroups table and an associated join table.

The database version for this release is 20160412030352.

Gem Installation

Administrators need to run the bundler to update versions of some of the Ruby Gems.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command

    bundle exec rspec

As noted above, we are now using RSpec version 3.4.

Complete instructions for setting up the test database and running the RSpec tests are on the Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 9 years ago

BETYdb - BETYdb 4.7

Upgrade of Rails and Enhancement of Documentation.

The Rails version we use has now been bumped up to version 3.2.22.2. This addresses certain security concerns. API version "v0" has been renamed to version "beta". The apipie Gem has been added, and code has been added to use this Gem to document the API and publish that documentation to <host>/<rails root path>/apipie. Finally, better documentation of the SchemaSpy database documentation feature has been added, and configuration settings may now be placed in the configuration file config/application.yml.

Summary of Changes

New Features

Configuration settings for SchemaSpy integrated with other configuration settings.

Configuration settings for building the SchemaSpy documentation of the BETYdb database schema may now be placed in the configuration file config/application.yml. Use of the configuration file lib/tasks/schemaSpyConfig.rb is deprecated. For more information, look at the comments in the file config/application.yml.template.

Apipie-rails Gem for API documentation

We now are using the apipie-rail gem to do inline documentation of the BETYdb APIs. Documentation is viewable at <host>/<rails root path>/apipie.

Steps Needed for Upgrade

Upgrading is straightforward:

In your command-line terminal, cd to the root of your Rails installation.
Do a git pull to get the latest code from GitHub.
Run bundle install to install new Gems.
Restart the Rails server (touch tmp/restart.txt if you are running under Apache).

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests.

Note that you will likely get deprecation warnings when you run the Rake tasks to set up the test database (or for that matter, any other Rake task).

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 9 years ago

BETYdb - BETYdb 4.6

Summary of Changes

New API and Use of Proper Datatypes

This release adds a new versioned API for adding new trait data and for querying most all data. It updates measurement and statistical data to use the proper datatypes and adds and modifies many features to make maintenance of BETYdb easier.

Administrators need to do a database migration.

See "Database Changes" below.

New API

A new API (version v0) has been added for obtaining data in both JSON and XML formats. The new API also supports insertion of new trait data (provided in either JSON or XML format), something not provided by previous APIs. Documentation of this API is forthcoming. (A brief introduction is given in some of the comments to GitHub issues https://github.com/PecanProject/bety/issues/381 and https://github.com/terraref/computing-pipeline/issues/52.)

New Features

Docker files for application packaging

Docker files have been added.

Travis

A Travis configuration file has been added in order to hook BETYdb into the Travis CI continuous integration service. This service will automatically run all of the non-Javascript based RSpec tests each time a new commit is pushed to GitHub.

New and Modified Rake Tasks

A new Rake task bety:db:populate has been added to make it easier for developers to get a new BETYdb instance up and running or to get a fresh copy of the BETYdb database. This is a wrapper for Pecan's load.bety.sh script and replaces the wrapper script update-betydb.sh for most use-cases relevant to the BETYdb developer.
db:test:load_structure has been modified to behave correctly when using the postgis adapter. It will now drop the test database if it exists and then recreate it and load the database schema defined by the *_structure.sql file corresponding to the current environment. This makes it easier to set up the test database for testing, even if the other databases have not been set up.
A description has been added for the Rake task db:test:clone_structure.
The task db:fixtures:load has been disabled for all but the test environment. This will help prevent data loss caused by inadvertently running it in the wrong environment.
The db:create task now only creates the database corresponding to the current environment. (When Rails.env was "development", it used to also create the test database. Though perhaps useful, this seemed counterintuitive.)

Bug Fixes

Some missing model associations have been added. This should facilitate getting useful information from related tables via an API call.
The script script/update-betydb.sh has been updated to make it consistend with the load.bety.sh script from the Pecan repository, for which it serves as a wrapper.
Some fixes have been made to the Add-SQL-Geometry Python script (script/buildSQLGeom.py).
A bug has been fixed in the header-normalization routine of the Bulk Upload wizard whereby any header (e.g. a trait name) containing the consecutive letters 'se' was getting capitalized. Now only the string 'se' itself is normalized (to 'SE'—Standard Error).

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

The migration ChangeNumericsToFloats changes the datatype of all columns that were previously arbitrary precision numbers of type NUMERIC(16, 4) into double precision floating-point types. The columns in question are those used to represent values of trait, covariate, and yield measurements and the values of associated statistics (and also the parameter columns in the priors table). This helps to make the number of significant digits recordable for a particular measurement independent of the unit used for recording the measurement. Moreover, as the PostgreSQL documentation points out, "arithmetic on numeric values is very slow compared ... to the floating-point types", so using floating-point types represents an improvement if computational features of SQL are to be used.

The database version for this release is 20160303221049.

Gem Installation

Administrators need to run the bundler to install new Ruby Gems related to the new API.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```
Pass/failure status of the latest commits to GitHub are now shown at https://travis-ci.org/PecanProject/bety. Only the non-JavaScript based tests are run by Travis. To check the others, run
```
bundle exec rspec -t js
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 9 years ago

BETYdb - BETYdb 4.5

Summary of Changes

This release implements a number of bug fixes and allows site managers to customize contact information for their site(s).

Administrators: Note important upgrade instructions below.

New Features

Customization

Site managers can now customize the contact information—the e-mail address and phone number—that appears in the footer of every page. This e-mail address is also used as an administrative contact e-mail address (additional to those found in the database's users table) for automatically-generated messages.

Bug Fixes

Partial Fix for Issue #332

For deployments where the URLs for the BETYdb site and the PEcAn site are http(s)://[Domain Name]/[bety path] and http(s)://[Domain Name]/pecan respectively, the "Results" links on the "Listing Workflows" pages should now work. (Note that bety path need not necessarily be the literal bety, but the PEcAn path must be pecan.)

Fix for Issue #355

It used to be that adding a new BETYdb user would result in an error if no administrative users (users with page_access_level = 1) had been added to the database because the application would attempt to send a sign-up notification e-mail to an empty list of administrators. The new contact information customization feature now ensures there is always at least one administrative e-mail address in the "To:" list, so this error will no longer occur.

Fix for Issue #362

The "CF Guidelines" link on the "New Variables" page should now work, even for sub-URI deployments. (Prior to this, the link only worked for root-path deployments such as https://www.betydb.org.)

Fix for Issue #373

Non-point sites used to give "NaN" as the altitude of the site because we return the coordinates of the centroid of the geometry for these sites, and the centroid is a 2-dimensional point. We now simply show the altitude as blank.

Partial Fix for Issue #374

Up until now, the BETYdb Web application only supported editing Point geometries. Moreover, updating a sites row via the Web app would silently convert non-Point geometries to Point geometries using the centroid function even if only fields unrelated to the geometry (or no fields at all) were updated!

This fix prevents the Web app from changing the geometry data of non-point sites in order to prevent data loss.

Fix for Issue #377

Existing users can now request changes to their own access levels.

Steps Needed for Upgrade

Custom Configuration

Once you have pulled the new code from GitHub and restarted your Rails server, the site should be up and running. But unless you create the new customization file config/application.yml, the footer of all pages will show bogus contact information in place of what has, up until now, been the hard-coded default (namely, phone number (217) 300-0266 and e-mail address [email protected]). Even more importantly, unless you have been using 'thisisnotasecret' as the value of REST_AUTH_SITE_KEY in the config initializer file config/initializers/site_keys.rb, none of your users will be able to log in.

To address these issues, you must create an appropriate YAML config file. Here are step-by-step instructions for doing so:

cd to the Rails root directory.
Copy the default configuration file to application.yml:
```
cp config/defaults.yml config/application.yml
```
Do a copy, not a move! You want to keep defaults.yml as is. (For more elaborate set-ups having per-environment configuration values, you may want to copy config/application.yml.template instead. These instructions are for the simplest case.)
Edit the string values for admin_phone and admin_email to the appropriate ones for your site.
Open config/initializers/site_keys.rb and find the value for REST_AUTH_SITE_KEY. If it is anything other than 'thisisnotasecret', copy the value and use it as the value of rest_auth_site_key in application.yml. Otherwise, simply delete (or comment out) the line in application.yml that sets rest_auth_site_key.
You now no longer need the file config/initializers/site_keys.yml, but don't delete it just yet! We want to be sure the site is working first: in case you mis-copied the site key value, we don't want the correct value to be lost forever. Instead, rename it:
```
mv config/initializers/site_keys.rb config/initializers/site_keys.rb-disable
```
Now the application will use and only use the value set in the application.yml file (or the value in defaults.yml if you aren't overriding the default value).
Restart your Rails server. If you are using PhusionPassenger with Apache, you can just do
```
touch tmp/restart.txt
```
Go to your site page. You should see your custom contact information in the footer. If you still see the old hard-coded values ((217) 300-0266 and e-mail address [email protected]), you may not have actually restarted the server.
Log out if you aren't already and then try logging in as any user. If you were able to log in, and you are sure that you actually did restart the server, it is now safe to delete config/initializers/site_keys.rb (which you renamed to config/initializers/site_keys.rb-disable above).

Status of RSpec Tests

There are now 246 RSpec tests that don't involve Javascript. These pass consistently, and may be run with the command
```
bundle exec rspec -t ~js
```
There are 28 RSpec tests that involve Javascript. Most of these pass consistently, but occassionally one of these tests will fail intermittently. (This may be a timing-related issue.) These tests may be run with the command
```
bundle exec rspec -t js
```

If one of these tests fails, the output will show the command needed to re-run the failing test. (If you aren't using RVM, you may need to prefix the command shown with bundle exec.) Often, the test will pass on a second try.

(Complete details for running the RSpec tests are on the Wiki pages at https://github.com/PecanProject/bety/wiki/Automated-Tests .)

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 9 years ago

BETYdb - BETYdb 4.4 now with access to twice as much data

Summary of Changes

This release implements a number of long-standing requests for changes to database tables, implements some new features, and fixes several bugs.

Administrators need to do a database migration. See "Database Changes" below.

Access to 'unchecked' data through web interface and API

Data are marked as 'unchecked' when they are first entered, and marked 'checked' after they have been independently compared to their source. QAQC is an ongoing effort, but most of the data are valid, so we have decided to make all unchecked data available. Previously, only checked data were available. Data flagged as incorrect are still marked as such and are are only accessible to administrators and data managers.

Unchecked data have previously been available to PEcAn users.

QAQC is a community effort - please report incorrect data as well as any data that you use and compare with an original publication or source of data, or another database / independent review.

New BETYdb chat room on Gitter

New Features

New Workflow Attributes

The Workflows model now includes a "notes" attribute and a link to the users table.

Advanced Search Option to View Unchecked Traits and Yields

The Advanced Search page now includes a check-box for optionally including unchecked data in the search results.

Link to New Gitter Chat Page

We're now on Gitter. A link to the Gitter chat page has been added to the footer. Just like the one above).

Option to Log Rake Tasks to STDOUT

The new log rake task can be used to redirect logging of subsequent tasks in the command-line task list to standard output. This is particularly useful when running migrations: By running the migration as

rake log db:migrate

the user can easily see what SQL commands were used to effect the migration.

Steps Needed for Upgrade

Database Changes (if applicable)

Administrators need to do database migrations!

There are three migrations associated with this release:

The first alters the view traits_and_yields_view so that unchecked rows are included.
The second adds new attributes notes and user_id (along with a foreign-key constraint for the latter) to the workflows table.
The third removes the dataformat attribute from the formats table.

The database version for this release is 20151914182146.

Status of RSpec Tests

There are now 246 RSpec tests that don't involve Javascript. These pass consistently, and may be run with the command
```
bundle exec rspec -t ~js
```
There are 28 RSpec tests that involve Javascript. Most of these pass consistently, but occassionally one of these tests will fail intermittently. (This may be a timing-related issue.) These tests may be run with the command
```
bundle exec rspec -t js
```

If one of these tests fails, the output will show the command needed to re-run the failing test. Often, the test will pass on a second try.

(Complete details for running the rspec tests are on the Wiki pages at https://github.com/PecanProject/bety/wiki/Automated-Tests .)

Bug Fixes

Fixed Typo in Mangements List

A typo in the management types list prevented fertilizer_other_rate and fungicide from appearing in the selection drop-down for choosing a management type in the New and Edit Management pages. This has been fixed.

Edit Species Displays Errors Properly

A validation error when attempting to edit a species was causing the "We're sorry, but something went wrong" page to be displayed. Now the user is correctly returned to the Edit page with a displayed list of validation errors.

Searching Formats Fixed

Searches on the "Listing Formats" were broken when the mime_type column in the formats table was replaced by a key into the mimetypes table. This has been fixed.

Show Pages for Runs Fixed

The show pages for runs was broken when the start_date and end_date columns were removed from the runs table. This has been fixed.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 9 years ago

BETYdb - BETYdb 4.3.1

This is a bug-fix release fixing a problem whereby inputs couldn't be created or edited if any format whatsoever lacked a mimetype. Also, in some cases, the list of inputs and/or the list of formats couldn't be displayed, and the show page for a format lacking a mimetype could not be displayed.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 9 years ago

BETYdb - BETYdb 4.3

This release implements some long-standing requested database changes, replaces awkward selection boxes used in some forms with autocompletion-enabled fields, and fixes some bugs with cloning of PFTs and generation of Priors images.

The primary motivation for this release was to implement a long-standing request to eliminate some unneeded database tables and columns (and the Rails machinery associated with them), replace the formats.mime_type column with a foreign-key column referencing the mimetypes table, and add some date and entity-related columns to the yields table similar to those that already exist in the traits table.

Along with this, several usability improvements and bug fixes have been implemented.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Autocompletion fields

Several of the Web forms for creating new entities and editing existing ones contained very lengthy selection lists for choosing (for example) the variable associated with a trait or the list of priors associated with a pft. These have been replaced with autocompletion fields so that typing characters into the field filters the list of options down to only those options containing that sequence in (one of certain designated columns of) the entity being selected.

Bug Fixes

The update to Rails 2 broke the on-the-fly generation of images associated with Priors. This has been fixed.
Cloning of PFTs would fail in some cases because we have imposed stricter validation of species objects. (For example, when you create a new species, the scientificname must textually include both the Genus name and the species name.) Many existing species rows fail the validation requirements. For this reason, we have turned off validation of associated species when cloning a PFT.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There is one migration in this release. It removes the inputs_variables, location_yields, and counties tables. It removes the start_date and end_date columns from the runs table. It replaces the mime_type column in the formats table with mimetype_id, a foreign key referencing the mimetypes table. And it adds four columns to the yields table: entity_id, date_year, date_month, and date_day.

The database version for this release is 20150904184512.

Status of RSpec Tests

All tests non-Javascript based tests pass. These may be run with the command
```
bundle exec rspec -t ~js
```
Most of the Javascript-based tests pass, but one of them seems to pass only intermittently.

The Javascript-based tests may be run with the command

 bundle exec rspec -t js

The test that sometimes fails begins at line 36 of the file spec/features/management_integration_spec.rb and may be run with the command

 bundle exec rspec ./spec/features/management_integration_spec.rb:36

It seems that the feature this test tests does in fact work, but an issue has been created to note the problem with this test.

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 9 years ago

BETYdb - BETYdb 4.2.2

The is a minor release to tie up some loose ends, but it also contains one substantial new feature--a script for inserting new managements from data fed in from a CSV file.

The primary new feature of this release is a new script, script/insert_managements.rb, which turns a CSV file of management data into a file of SQL commands for inserting new managements and associated managements_treatments entries.

In addition, this release contains many refinements to the the bety:dbdocs rake task that produces the database schema documentation viewable within the BETYdb Rails application making it easier to use, and it removes from the Git repository the several files generated by this script. And it contains one minor bug fix.

Changes Pertinent to PEcAn Users (if applicable)

There are no migrations included in this script. Users who want to have a working copy of the database schema documentation, however, will have to generate the schema documentation with the new rake bety:dbdocs task since the (outdated!) files previously used have been removed from the Git repository.

Summary of Changes

New Features

Insert New Managements Script

The new script is called insert_managements.rb. Full instructions and documentation of its use may be had by running the command

bundle exec script/insert_managements --man

from the Rails root directory. (The bundle exec prefix is not necessary if using RVM.)

Refinements to the bety:dbdocs Rake Task

It is now possible to set the java command to use for generating the schema documentation in the task configuration file. If not set, it defaults to "java".

Bug Fixes

GH Issue #293: Pre-filled-in species name shouldn't disappear in Cultivar update form when a validation error occurs.

This is now fixed.

Steps Needed for Upgrade

Database Changes

No database migrations are required.

The database version for this release is (still) 20150625184958.

Gem Installation

Administrators need to run the bundler to install the trollop Gem.

This Gem is required by the new insert_managements.rb script.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.2.1 Critical update to fix SQL constraints

This is a critical update to correct SQL functions; it also includes miscellaneous other changes.

The primary motivation for this release is to correct id-getting SQL functions that are being used to implement a distributed foreign-key constraint on the dbfiles table. Without this fix, non-EBI users will be unable to import data. A database migration corrects the problem functions.

Two other migrations are included: The first adds columns to the machines table having to do with data synchronization. The second implements default values on timestamp columns which historically have often been left as NULL.

Also included is a major feature addtion: the ability to generate and update database schema documentation using a rake task that runs SchemaSpy.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Rake task to generate database schema documentation

Database schema documentation is viewable from the Rails app by selecting "Schema" in the "Docs" menu. But up until now, this documentation has not been kept up to date. Now, up-to-date documentation can be generated using the "rake bety:dbdocs" task.

Bug Fixes

The above-mentioned migration fixes a problem with SQL functions that would prevent users from importing data to their own copies of the BETYdb database (GH issue #327).
A long-standing error in the way the traits list page was displayed for users with "Viewer" status has been fixed (GH issue #299).

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There are three migrations in this release (see descriptions above).

The database version for this release is 20150625184958.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.2 New Value Constraints

New value constraints for database tables and validations for corresponding models; new RSpec tests; several bug fixes.

Many new values constraints have been added to ensure database-integrity. Consistency checks have been added to the dbfiles table to ensure the referred-to containers exist. Restrictions on editing variables associated with traits or covariates that violate the range restriction for the variable value have been relaxed. Many bug fixes have been implemented.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

Implemented values constraints, both at the database level and and the Web user-interface (Rails app) level

The addition of value constraints is a major step toward helping to ensure data-integrity. It should no longer be possible, for example, to enter a site with a longitude value of 2000 (which has happened!) or genuses that aren't capitalized. Not only will these constraints help prevent gross errors, they will also help standardize the set of allowable values. When used to constrain values in columns comprising a candiate key, this will make it less likely that duplicate rows for the same entity will occur. To give one example, a uniqueness constraint on species.scientificname will not by itself prevent the occurrence of two species rows that are identical except that the first has scientificname "Abies alba" and the second has scientificname "Abies alba ". Requiring values in this column to be "whitespace-normalized" (no leading or trailing space, no double-spaces) will prevent this sort of duplicate from occurring.

Implemented integrity constraints on references to inputs, models, and posteriors in dbfiles table

A row having container_type = 'Input' must have a value for container_id that matches the id column value of some row in the inputs table. Similarly for container types "Model" and "Posterior".

Allowing "lazier" approach to fixing variable range violations

It used to be a variable could not be edited if any trait or covariate row that referred to the variable had a variable value that violated the range restriction for the variable. One first had to fix all the range violations in the traits and/or covariates rows before the variable could be edited, even if the update only involved, say, changing the notes. Now the trigger that checks for range violations is only run when an attempt is made to change the value of the min or max attribute. Thus all other attributes are now freely editable, regardless of any range violations in associated rows.

Bug Fixes

Some of the fixes include:

Corrected text of mis-labelled buttons
Fix for issue #282: Can't create two new managements in a row for a given treatment.
Eliminated duplicate fields from New Species form
Fixed layout of Species and Covariate "Edit" pages (GH #280)
Fixed trait-search action used on Covariate "Edit" page
Eliminated vestigial obsolete code and routes
Restored the change-password checkbox to the edit-user page to make this page clearer and more user-friendly
Re-aligned and rearranged fields in species/show page (GH #135)
Restored some routes that were eliminated when wildcard routes were eliminated.
Corrected title for Machine "Show" page (GH #294)
Allow Managers to delete Yield records; also allow Creators to delete Yield records they themselves created (part of Redmine issue #2334)
Made treatments on Listing Treatments page sortable
Restyled Listing Treatments page to make it more readable, and added delete buttons to rows that user is allowed to delete (part of Redmine issue #2334)
Better error message when treatment is missing in bulk upload file (GH #287)
Better error message for violation of foreign key constraint (GH #306)
Fixed "CF Guidelines" link on Variables "Edit" page (GH #308)
Fixed Variable update form so that variable type can be updated (GH #309)
Fixed Management "Edit" page so that citation is not inadvertently changed (GH #313)
Fixed "map" links on Site listing page that were broken in certain deployment configurations and removed "map" link from "Show" pages (GH #305)
On "Edit" pages for traits and yields, preselect the current access level so that it isn't inadvertently changed (GH #302)
Fixed display of incorrect QA/QC values on yields listing page (part of GH #303)
Fixed bug in data access filter so that Views and Creators can once again see all records that they themselves created
Changed criteria for displaying the edit control for QA/QC values on the yields list page (part of GH #303)
Re-styled "New File" page (GH #304)
Removed "Show" button from "New Site" page (GH #321)
Changed "Back" buttons on all "New" pages to say "All Records" like all the edit pages do (GH #322)

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

There are three database migrations included in this release. The first and most extensive of these involves the addition of several new value constraints. The following set of SQL commands may prove helpful in applying this migration, especially if a data synchronization has not been done recently:

update ensembles set notes = '', updated_at = now() at time zone 'utc' where notes is null;
update runs set outprefix = '', updated_at = now() at time zone 'utc' where outprefix is null;
update runs set setting = '', updated_at = now() at time zone 'utc' where setting is null;
update users set state_prov = '', updated_at = now() at time zone 'utc' where state_prov is null;
update users set postal_code = '', updated_at = now() at time zone 'utc' where postal_code is null;
update workflows set params = '', updated_at = now() at time zone 'utc' where params is null;

If further trouble-shooting of migrations is required, the following may be helpful in finding the source of migration problems:

bundle exec rake db:migrate 2>&1 | grep ERROR

(If you run BETYdb in the production environment, and your production database differs from you development database, prefix this command with RAILS_ENV=production.)

The database version for this release is 20150521211114.

Gem Installation

No new Gems need be installed for this release.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.1 New Database Uniqueness Constraints

New uniqueness constraints for database tables and validations for corresponding models; new RSpec tests; several bug fixes.

This release implements several value and uniqueness constraints on database tables aimed at ensuring that the BETYdb database will not contain multiple rows describing the same citation, site, species, treatment, or other entity. In conjunction with the database changes, new model validations have been added to several of the models so that potentially invalid database insertions or updates are caught at the Rails level before the insertion or update is attempted on the database.

Second, several bugs existed that prevented access to many pages of the site--especially pages for editing database entities. Many of these were introduced by changes to the routing file that eliminated most wildcard routes. These bugs have now been fixed and several new tests have been added to ensure these bugs don't re-appear.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

New Validations to Prevent Duplicate Rows and Missing Data

Bug Fixes

Many pages that were previously inaccessible are once again reachable. Prior to this update, attempting to visit these pages yielded the following error:

The page you were looking for doesn't exist.

You may have mistyped the address or the page may have moved.

Affected pages included (for example)

the Covariate editing page
the Model editing page
the Modeltype editing page
the Species editing page
the Traits editing page

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

One migration has been added that new value and uniqueness constraints and also adds some convenience functions for implementing these constraints.

The database version for this release is 20150202220519.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Every effort has been made to make this command idempotent: You should be able to run the tests multiple times without reloading the test fixtures between runs.

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.0.2 Bulk Upload Manager Access

minor release

This is a minor release that fixes a bug in the Bulk Upload feature and gives managers access to the validation portion of the Bulk Upload wizard.

Summary of Changes

Feature Additions

Users classified as managers (page access level 2) are now able to access the Bulk Upload wizard. They can upload and validate CSV data files but are not permitted to insert the data they contain. This is the first expansion of access to the Bulk upload feature beyond core developers.

Bug Fixes

A minor bug in the Bulk Upload feature was fixed. Missing citation_year values used to trigger a fatal error, but they no longer do.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.0.1

This is a minor release that extends the API search syntax to the traits_and_yeidls_view and fixes some bugs in the Trait and Yield creation and update pages. It was deployed to www.betydb.org on April 30, 2015.

Summary of Changes

Feature Additions

It is now possible to get query results from the traits_and_yields_view SQL view using multiple columnname=value clauses in the query string appended to the Advanced Search page URLs https://www.betydb.org/search.xml (for results in XML format) and https://www.betydb.org/search.json (for results in JSON format). For example, the URL https://www.betydb.org/search.xml?lat=42.9&lon=-72.3&result_type=traits will find all checked trait data for sites located at 42.9 deg N latitude and 72.3 deg W longitude to which the user has access rights and return it in XML format. Note that matches must be exact! These strict search clauses may be combined with the less-strict searching provided by a clause of the form search=term1+term2+term3... which requires only that each of the terms provided be textually included in the column value of one of the columns sitename, city, scientificname, commonname, author, trait, trait_description, or citation_year.
A new rake task has been added as a convenience for displaying the database configuration file database.yml. Run
```
rake dbconf
```

Bug Fixes

A long-standing bug in the processing of the New Yield and Editing Yield forms rendered the Julian date fields useless. This has been fixed.
In addition, the upgrade to Rails 3.2 produced some new problems with the way the standard date fields in the New Yield and Editing Yield forms were processed. Furthermore, when the New Trait and New Yield forms were visited with a row id parameter passed in the URL, the forms would be pre-populated as before, but pressing the Create button would do an update rather than creating a new row as it was supposed to. These problems have been fixed as well.
Finally, API queries involving an include[]=<table> clause now once again show information for the included table in the query results. Even with this, queries work slightly differently than they did prior to the update to Rails 3.2. Details will be given in the BETYdb Data Access document at https://www.authorea.com/users/5574/articles/7062.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 10 years ago

BETYdb - BETYdb 4.0: enhanced API for rOpenSci, Rails 3.2, Ruby 2.1.5 upgrade

Enhanced API for rOpensci traits package; upgrade to rails 3.2, Ruby 2.1.5.

This release upgrades the Rails version from Rails 3.0.18 to Rails 3.2.21 and changes the RVM-specified version of Ruby from version 1.9.3-p429 to version 2.1.5. In addition to these changes which in and of themselves make for improved security, there are numerous minor changes to the Rails code including security fixes, removal of obsolete code, fixes and improvements to the query API, and phase-out of vendor plugins in favor of gems,

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

New Features

Enhanced API for rOpensci traits package

See section 2 of the BETYdb Data Access guide at https://www.authorea.com/users/5574/articles/7062.

API Key generation

Users may now generate their own API keys by going to the Users page and clicking the "key" button. And there is now a new rake task that administrators can use to generate API keys for all current users. See section 2.3 of the BETYdb Data Access guide at https://www.authorea.com/users/5574/articles/7062.

Directory-Specific Session Keys

When multiple copies of BETYdb are deployed on the same host, they now use differently-named session cookies (assuming the basenames of the Rails root directories are different). This mainly affects developers who may visit different copies of BETYdb on the same host simultaneously; up to now, such users will likely have seen baffling interactions between the two copies.

Updated version of selenium-webdriver

The Gem version specification for selenium-webdriver has been changed to a version compatible with the latest versions of Firefox. This mainly affects those running the RSpec tests in debug mode.

Addition of primary keys to join tables

This facilitate the synchronizing of data having to do with many-to-many associations.

Security Updates

Miscellaneous security updates have been implemented, including

Upgrading of Rails from version 3.0.18 to 3.2.21
Upgrading the default version of Ruby to version 2.1.5
Storage of session data in the database instead of in cookies
Elimination of most non-specific routes

In keeping with recommended practice in Rails 3.2, packages that were formerly kept in vendor plugins are now in Ruby gems.

The change to database storage of session data has the additional benefit of overcoming some restrictions on the size of bulk uploads.

Bug Fixes

The API for querying and downloading data offers more features than available previously, restoring some features that were lost in the upgrade from Rails 2 to Rails 3. See section 2 of the Data Access Guide: https://www.authorea.com/users/5574/articles/7062
Cloning of PFTs now works again.
A bug that prevented editing a PFT if it was associated with some Prior whose paramb column was NULL has now been fixed.

Steps Needed for Upgrade

Database Changes

Administrators need to do database migrations!

One migration has been added that adds id columns to the join tables.

The database version for this release is 20150202215147.

Ruby Upgrade

The officially-supported version of Ruby for use with BETYdb is now Ruby version 2.1.5. It is recommended that you upgrade to some Ruby 2 version since Ruby 1.9.3 is no longer being maintained. (See https://www.ruby-lang.org/en/news/2015/02/23/support-for-ruby-1-9-3-has-ended/ for details.)

Nevertheless, users who can not or do not wish to upgrade will likely be able to continue to use Ruby 1.9.3 without difficulty. It may also be possible to use the latest version of Ruby, Ruby 2.2.1.

To continue using Ruby 1.9.3

If you use RVM or some other Ruby version management system that reads the .ruby-version file, revert the string in that file so that it once again reads ruby-1.9.3-p429. You may wish to add .ruby-version to your .gitignore file so that git doesn't try to change it or flag it as having been modified.

To upgrade to Ruby 2.1.5

Install Ruby 2.1.5 if you haven't done so already. If you use RVM, you may do this by running

rvm install 2.1.5

If you run BETYdb with Apache httpd, change or add the directive

PassengerDefaultRuby /path/to/ruby2/executable

or the

PassengerRuby /path/to/ruby2/executable

directive according to your needs.

Otherwise, if you only use WEBrick, just ensure that the version of Ruby you want to use is first in your path. If you are using RVM, this is automatic (though you may have to leave and then return to the BETYdb Rails root directory if you haven't done so since updating your .ruby-version file; run rvm current to verify that you are using the correct Ruby version and gemset).

Gem Installation, Including Rails Upgrade

Administrators need to run the bundler to install several new Ruby Gems and updated versions of existing ones.

First, change to the root directory of your BETYdb Rails installation. Then, if you are using RVM, run rvm current to ensure you are using the correct Ruby version and gemset. It should say ruby-2.1.5@betydb_rails3 if you upgraded Ruby and ruby-1.9.3-p429@betydb_rails3 if you did not. If you are not using a Ruby version management system, simply run ruby -v to check your version of Ruby.

Once you ensure you are using the correct Ruby version and gemset, run

bundle

If you have trouble installing capybara-webkit and don't need to run the Javascript-based RSpec tests, you may run

bundle --without javascript_testing

instead. If you won't be running RSpec tests at all and want to minimize your Gem installation, run

bundle --without javascript_testing test debug

See https://github.com/PecanProject/bety/wiki/Installing-the-BetyDB-Rails-Application#minimizing-gem-installation for details.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde about 10 years ago

BETYdb - BETYdb 3.3.2 Foreign Key Constraints

This release primarily implements several foreign-key constraints on the database. It consists of a new migration file (together with an updated version of production_structure.sql), some related changes to the controller code, and an unrelated minor change consisting of an update to a link in the footer and removal of no-longer-needed routing code.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

See "Database Changes" below.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20141211220550.

Gem Installation

No new gems need be installed.

Additional Notes

Other than the addition of a migration, the primary change to the Rails code is some error-handling code to gracefully handle foreign-key constraint violations and provide users with helpful error messages. Currently, these message are in most cases very rudimentary but nevertheless may be comprehensible to non-programmers.
Some unneeded but misleading Rails code was removed. (This is code which looks like it does something but does nothing, so its deletion should not be noticed by users.)
The target of the Documentation link in the footer is now the BETYdb wiki instead of the PDF version of the Database Description and User's Guide.
The migration will fail if your copy of the bety database violates any of the new foreign-key constraints. The master database on ebi-forecast, however, has been updated to comply with this constraint, so it is recommended to update your copy of the database to mirror the changes made there.

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```

One bulk upload-related test has been marked 'pending', however, until it is reconciled with the new constraints.

Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.3.1

This release implements changes to the database schema discussed in Github issue #145 and makes a few related changes. It consists of a single migration file together with an updated version of production_structure.sql.

Changes Pertinent to PEcAn Users

Administrators need to do a database migration.

[The BETYdb web interface, however, will, for now, continue to function as before even without doing the pending migration.]

See "Database Changes" below.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20141208165401.

A new table cultivars_pfts has been added so that PFTs consisting of cultivars rather than of species may be formed.
Uniqueness, foreign-key, and non-null constraints have been added to this new table and to the existing pfts_species table.
pfts_species and cultivars_pfts have been constrained in such a way that a PFT may be associated with a set of species or with a set of cultivars but not with a set containing both species and cultivars.
A uniqueness constraint has bee added to pfts to ensure that names are unique within the set of PFTs of a given model type. (PFTs associated with different model types, however, may have the same name.)

Additional Notes

Other than the addition of addition of a migration, the Rails code is essentially unchanged. In particular, no facility has yet been added in the Rails code to work with PFTs consisting of cultivars.
The foreign key constraints have been designed in such a way that a PFT may be removed without having to first empty it of species or cultivars. This is as before, except that now, all entries in the join tables pfts_species and cultivars_pfts that refer to the removed PFT are automatically removed as well.
Species and cultivars, however, may not be removed if they belong to any PFT. They must first be removed from all PFTs containing them before they themselves can be removed.
The migration will fail if your copy of the bety database violates any of the new uniqueness constraints. In particular, pfts are now required to be unique by (name, modeltype_id). The master database on ebi-forecast has recently been updated to comply with this constraint, so it is recommended to update your copy of the pfts table to mirror the changes made there. A query that can be used to find the offending rows is
```
select array_agg(id) as id_list, name, modeltype_id, count(*) from pfts group by name, modeltype_id having count(*) > 1;
```
Most likely, this will produce only a row or two, each with an array of pft id numbers containing only two or three entries. Thus only two or three rows of the pfts table need be updated which can easily be done manually. (I found it sufficed to update the name in row 83 to 'plants2' and the name in row 95 to 'plants3'. Visit the PFT pages at https://www.betydb.org to see what the name for any particular id number has been changed to in the master database. For example, to see the current name used for the PFT with id=95, visit https://www.betydb.org/pfts/95.)

Status of RSpec Tests

All tests continue to pass when run in the default environment and can be run using the command
```
bundle exec rspec
```
Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.3

This incorporates a major new feature--the Bulk Upload Wizard--and also includes bug fixes from the last several sprints. It was deployed to www.betydb.org on December 5, 2014.

Major Changes

Administrators either need to run bundle install to install several new
Ruby Gems or configure bundler to ignore the Gems in the 'debug' group. (The
'debug' group is needed only for running RSpec tests using the Selenium
JavaScript driver.) Also, to avoid the installation of capybara-webkit, the
group to ignore is now called 'javascript_testing', so if you never installed
it and don't want to install it, configure Bundler to ignore the
'javascript_testing' group.

(Note: Unless you have difficulty installing the Gems used for testing, there is
really no compelling reason not to install them.)

To ignore the 'debug' group, either run

bundle install --without debug

or, if you are sure your other Gems are up-to-date, you can just run

bundle config --local without debug

To ignore the 'javascript_testing' group (thus avoiding installation of
capybara-webkit), substitute 'javascript_testing' for 'debug' in either of these
two commands. To ignore both, run

bundle install --without debug javascript_testing

or

bundle config --local without debug:javascript_testing

Or, to ignore ALL gems except those needed in the production environment, ignore all the other groups:

bundle install --without debug javascript_testing test development

or

bundle config --local without debug:javascript_testing:test:development

Note that bundle install --without takes a space-separated list, but bundle config without takes a colon-separated one.

Summary of Changes

New Features

A wizard for bulk upload of data has been added (currently only available to administrators).
An easy method has been provided to debug RSpec tests using the Selenium JavaScript driver together with breakpoints (see the updated Wiki page for Automated Tests).
Distinct pages now have distinct titles, making the site more accessible for those using screen readers and making navigation of the site using the browser history possible.

Minor Improvements

Redmine issue #2599 ("Different pages of BetyDB should have different
titles."): Formerly, every page on the site had the same title ("Energy
Biosciences Institute"). This meant that if one had been clicking around the
site for awhile and then looked at the browser history, all the recent entries
in the history list would appear to be exactly the same. Now the default
title is "BETYdb: " followed by the path/filename portion of the URL of the
page.
New validations have been added to the Create and Update pages for some
tables. For example, it is now checked that yield values are non-negative.
The rake notes:todo task has been redefined so that it finds not only
source-code comments beginning with "TODO", but also those beginning with
"TO-DO" or "to-do".

Bug Fixes

These bugs have been fixed:

Redmine issue #2406 ("Searching is broken on Ensembles, Inputs, Runs, and
Workflows pages.") Searches on these pages have now been fixed.
Redmine issue #2486 ("Baffling trait-creation error.")
Redmine issue #2604 ("Application error when unsuccessfully updating a yield."

Status of RSpec Tests

All tests now pass when run in the default environment and can be run using the command
```
bundle exec rspec
```
Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Automated-Tests

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.2.4

This incorporates some features and bug fixes from the September 29, 2014 sprint. It was deployed to www.betydb.org on October 9, 2014.

Changes Pertinent to PEcAn Users

Administrators need to do database migrations!
See "Database Changes" below.

A constraint was added to the geometry column of the sites table to ensure the geometry is valid.

Release Highlights

The update.psql.sh script has been updated to incorporate changes to the corresponding pecan script—namely, it now adds a set of default users having all possible permission levels and adds the guestuser account.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20141009160121.

A constraint was added to the geometry column of the sites table to ensure the geometry is valid.

Minor Changes and Improvements

Contact information on the home page has been updated.
Instructions for citing BetyDB have changed.

Bug Fixes

Redundant error messages are no longer displayed on the home page when login fails.

Status of RSpec Tests

All tests now pass when run in the default environment and can be run using the command
```
rspec
```
Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Testing

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.2.3

This adds a migration, restoring some defaults for column values that were lost in the migration from MySQL to PostgreSQL. It was deployed to www.betydb.org on September 19, 2014.

Summary of Changes

Database Changes

Administrators may need to do database migrations if they wish to sync their database with the one on ebi-forecast.

The database version for this release is 20140915153555.

This is a very minor migration: It sets three default values:

a default of 0 for the checked column of the traits table
a default of 0 for the checked column of the yields table
a default of '' for the name column of the users table

This restores defaults that were set in the MySQL version of the database before it was migrated to PostgreSQL.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.2.2

This adds a migration, adding a new table to the database. It was deployed to www.betydb.org on September 9, 2014.

Changes Pertinent to PEcAn Users

Administrators need to do database migrations!
See "Database Changes" below.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140909212759.

A new table, trait_covariate_associations has been added. This will be used to facilitate bulk uploads of trait data.

Status of RSpec Tests

All tests now pass when run in the default environment.
(Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Testing)

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.2.1

This incorporates fixes for two bugs discovered during the September 5, 2014 sprint. It was deployed to www.betydb.org on September 5, 2014.

Changes Pertinent to PEcAn Users

Administrators need to do database migrations!
See "Database Changes" below.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140904221818.

The user_id columns of the modeltypes and modeltypes_formats tables have been changed to big ints to match their referent.
The views which use the geometry column of the sites table have been updated so that they behave correctly when the column value is a polygon rather than a point.

Bug Fixes

The change to the views fixes the advanced search page, which has been broken since polygons were entered into the sites table.

Status of RSpec Tests

All tests now pass when run in the default environment.
(Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Testing)

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.2

This incorporates the features and bug fixes from the Augsut 20, 2014 sprint and a few from the July 18 sprint. It was deployed to www.betydb.org on August 27, 2014.

Changes Pertinent to PEcAn Users

Administrators need to do database migrations!
See "Database Changes" below.

Creation and modification of ensembles, posteriors, runs, and workflows is no longer possible through the BetyDB web interface. This now must be done through PEcAn.

Release Highlights

The pages for adding and editing traits has been improved with a more logical layout of the attribute fields; and the panel for editing trait covariates has been moved to a side bar.
Users now have the ability to edit information for variables associated with formats.
New model types can be created in BETY and can be associated with PFTs as well as models.
Model types can now have a list of required inputs needed to run the model.
The "New" and "Edit" pages for ensembles, runs, and workflows have been removed as this functionality has been passed off to the PEcAn software.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140729045640.

A new modeltypes table has been added, and the textual model_type columns in the models and the pfts tables have been replaced by references to rows of the new table.
A new modeltypes_formats table has been added; this allows one to specify what file formats are needed for execution in PEcAn as well as what tags are used to specify these files as inputs in the pecan.xml file.

Minor Changes and Improvements

P and HSD have been removed from the list of stat names that may be selected when adding or editing traits or yields.
The interface for associating variables with a format has been improved.

Bug Fixes

The Willow map legend has been fixed.
The code that generates images for the priors listing now references the correct version of R.

Status of RSpec Tests

All tests now pass when run in the default environment and can be run using the command
```
rspec
```
Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Testing

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.1 hotfix 1

This release applies an urgent fix to release BETYdb 3.1 to fix site searches.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.1

WARNING WARNING

The migration in this release will potentially remove some information. If you have sites that have a latitude and longitude, but no elevation, the latitude and longitude will be lost! You can check to see if you are affected by running the following SQL statement: select id, lat, lon from sites where lat is not null and masl is null; all sites returned are affected. To fix these sites (and set the elevation to -999, you can run the following command update sites set masl=-999 where lat is not null and masl is null;. You will have to execute this BEFORE the migration

WARNING WARNING

This incorporates the features and bug fixes from the July 18, 2014 sprint. It was deployed to www.betydb.org on August 1, 2014.

Major Changes!

Administrators need to do database migrations, update their database.yml file, add the PostGIS extension to their database and possibly install new packages to support PostGIS!

In the best case scenario, the following steps should suffice to carry out the upgrade:

Add the PostGIS extension to your PostgreSQL installation if needed.
Run psql -d bety -c 'CREATE EXTENSION postgis;' to add PostGIS to your copy of the bety database. (If you aren't yourself a superuser for your PostgreSQL installation, add the -U <some superuser> flag to this command, e.g. -U postgres. Or possibly run using sudo: sudo -u postgres psql ....)
If you wish to be able to run RSpec tests, also add the PostGIS extension to template1. (Repeat the above command with "template1" in place of "bety".)
Modify your config/database.yml file by replacing each "adapter: postgresql" line by "adapter: postgis". Also, if you created a section for the test_js environment, you may eliminate it. This environment is no longer used.
Run the bundler to install the new PostGIS-related gems: bundle install (Note: If you have trouble installing the RSpec test-related gems (capybara-webkit in particular can be difficult), you can run the bundler with the --without option to skip gems only needed by the test environment: bundle install --without test. This is a "remembered" option, and to undo it, you will need to run bundle config --delete without. Alternatively, you can simply comment out problem test-related gems in your copy of the Gemfile. For example, if capybara-webkit installation is a problem, simply comment out that line. You will still be able to run RSpec on the tests that don't require it. See the testing wiki at https://github.com/PecanProject/bety/wiki/Testing for details.)
Run the database migrations for your development and/or production database:
```
rake db:migrate
```
and/or
```
RAILS_ENV=production rake db:migrate
```
Restart your rails server (or run touch tmp/restart.txt if running under apache).

Release Highlights

The advanced search page now supports searching trait and yield data by the author or year of the associated citation.
The database and Rails software now supports PostGIS.
All rspec tests now pass when run in the default testing environment, which now includes capybara-webkit for running Javascript-related tests. The test_js environment is no longer supported. Testing instructions have been greatly simplified. See the testing wiki at https://github.com/PecanProject/bety/wiki/Testing for details.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140708232320.

The database now uses the PostGIS extension.
The the lat, lon, masl, and espg columns have been eliminated from the sites table. Location information is now represented by the new "geometry" column, which has type "geometry", at type only available under PostGIS.

New Features

Non-administrative users can now edit their own user profile.

Minor Improvements

The "Sites >>" link has been removed from the Traits and Yields listing pages since it doesn't fit with the usual data-entry workflow.
The species view page has been correctly styled to be consistent with other pages of the site and has been greatly simplified.
The Sites list page has been updated to be consistent with other list pages, combining the various action controls into a single "Actions" column, and using buttons rather than links for the controls.
Associated covariates (if any) are now listed on the trait show page.
Unlinking covariates from a trait now works correctly and is much easier.
Statistics for traits and yields now appear in a single column in an easy-to-read form on the traits and the yields index pages.
Numerous RSpec tests have been added.
The feedback tab has been made less obtrusive.

Bug Fixes

Minor problems causing some of the RSpec tests to fail when run with capybara-webkit enabled have been fixed. All tests now pass when run in the default "test" environment.
The Workflows listing page is now viewable.
The Methods listing is now viewable by both Administrators and Managers, and the link to the listing does not appear for users with lesser privileges.

Status of RSpec Tests

All tests now pass when run in the default environment and can be run using the command
```
rspec
```
Complete details for running the rspec tests are on the updated Wiki page at https://github.com/PecanProject/bety/wiki/Testing

Renewable Energy - Bioenergy - Ruby
Published by gsrohde over 10 years ago

BETYdb - BETYdb 3.0.5.2

This incorporates the features and bug fixes from the July 1, 2014 sprint. It was deployed to www.betydb.org on July 7, 2014.

Release Highlights

Allow users with manager access or higher to use provenance tables (RM 2007)
Data editors can now include site and treatment information when adding yields data.
A Feedback tab has been added conspicuously to all pages to allow easy submission of feedback.

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140624185610

schema.rb has been deprecated, is no longer part of the git repository, and is no longer automatically created by various rake scripts.
sites.mat now has type numeric(4, 2) instead of being an integer.
The pfts table has a new model_type column.
A join table for posteriors and ensembles has been added.

New Features

A Feedback tab has been added conspicuously to all pages to allow easy submission of feedback.
Users with manager access or higher are now allowed to use provenance tables (RM 2007)
Data editors can now include site and treatment information when adding and editing yields data.

Minor Improvements

The sites lists on the inputs forms are now sorted.
The Runs menu now appears in sorted order.
Trait covariates are now listed on the trait Show page.
Links from the treatments page to the traits and yields pages now go to the trait and yield listings rather than to the forms for new traits and yields.

Bug Fixes

Generation of images for the Priors listing has been fixed.
Posteriors can no longer be edited through the BetyDB web interface.

Status of RSpec Tests

(Generally we don't release code until all RSpec tests pass, but we have made an exception in this case.)

All tests pass in the default test environment except "Posteriors search search on posteriors should show search result on current page" at line 18 of spec/features/posterior_search_spec.rb; this is because this test requires the capybara-webkit gem, which isn't included in the "test" environment. This should pass in the "test_js" environment, which does include this gem, but there is currently a routing error bug involving the jQuery libraries that causes this and several other tests to fail when run in the test_js environment.

Renewable Energy - Bioenergy - Ruby
Published by dlebauer almost 11 years ago

BETYdb - BETYdb 3.0.3.9 hotfix 1

This release applies an urgent fix to release BETYdb 3.0.3.9 to allow persons entering yield data to set the site and treatment information for a yield entry.

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 11 years ago

BETYdb - BETYdb 3.0.3.9

This incorporates the features and bug fixes from the June 17, 2014 sprint. It was deployed to www.betydb.org on June 20, 2014.

Release Highlights

New links to updated documentation, moved key documents to Authorea; added schema navigation using SchemaSpy (#43).
New user requests sent to all users with administrative privileges (#56, RM #1707)
Updated database schema. See below

Summary of Changes

Database Changes

Administrators need to do database migrations!

The database version for this release is 20140617163304.

New fields in variables table to support cf-style variable naming (#59)
Added pft type to pfts table
Added posterior_samples, projects, and current_posteriors tables
Removed posteriors_runs table
Removed model_path from models table

New Features

A schema browser has been added to conveniently show the database structure.
Configuration changes make it now possible to run under Ruby 2.

Minor Improvements

More intuitive navigation: back buttons have been replaced by "All Records" buttons that go to a page showing all the records of the type being shown or edited
Added tests
- Added capybara webkit gem to support JavaScript-related tests; see notes in deployment wiki.
Restyled pages

Bug Fixes

Fixed search function on Posteriors and Managements pages (RM #1908 and #1922)
New User form now shows the "reason" input area when user requests elevated access (RM #2085)
It is now possible to edit site and treatment on the Traits edition page (RM #1881)

Renewable Energy - Bioenergy - Ruby
Published by gsrohde almost 11 years ago

BETYdb - BETYdb 1.6

Rails 3 upgrade

Renewable Energy - Bioenergy - Ruby
Published by dlebauer almost 11 years ago

BETYdb - BETYdb 1.5

First draft of new styling pages; updated maps.

Released for EBI Fall reviews 2012

Renewable Energy - Bioenergy - Ruby
Published by dlebauer almost 11 years ago

Recent Releases of BETYdb

BETYdb - BETYdb 5.4.1

BETYdb - BETYdb 5.4.0

Added

Fixes

Changed

BETYdb - BETYdb 5.3.0

Password encryption

This release requires a migration.

Steps Needed for Upgrade (Non Docker)

Steps Needed for Upgrade (Docker)

PEcAn dump script

Complete list of changes

Fixes

Added

BETYdb - BETYdb 5.2.2

Changed

BETYdb - BETYdb 5.2.1

Added

Changed

Fixed

BETYdb - BETYdb 5.2.0

Steps Needed for Upgrade

BETYdb - BETYdb 5.1.0

Fixes

BETYdb - BETYdb 5.0.5

Fixes

BETYdb - BETYdb 5.0.4

Fixes

BETYdb - BETYdb 5.0.3

BETYdb - BETYdb 5.0.2

Update of Sprockets Gem Version

Summary of Changes

Steps Needed for Upgrade

BETYdb - BETYdb 5.0.1

Fix for API fuzzy matching

Summary of Changes

Steps Needed for Upgrade

BETYdb - BETYdb 5.0

Upgrade to Rails 5

Summary of Changes

Bug fixes

Steps Needed for Upgrade

BETYdb - BETYdb 4.21

Add Foreign-Key Constraints and Database Maintenance Scripts

Summary of Changes

New features

Bug fixes

Steps Needed for Upgrade

Status of RSpec Tests

BETYdb - BETYdb 4.20 Upgrade to Rails 4.2

Upgrade to Rails 4.2; several bug fixes.

Summary of Changes

New features

Bug fixes

Other Changes

Steps Needed for Upgrade

Status of RSpec Tests

BETYdb - BETYdb 4.19

Publication of the beta API as version 1 (/api/v1); major speed-up of Bulk Upload wizard and several bug fixes.

Summary of Changes

Bug fixes

Other Changes

Steps Needed for Upgrade

Gem Installation

Status of RSpec Tests

BETYdb - BETYdb 4.18

Implements API query of sites by geographical coordinates; fixes constraints and sequence numbering of sites_cultivars table

Changes Pertinent to PEcAn Users

Summary of Changes

New Features

Bug fixes

Steps Needed for Upgrade

Database Changes

Gem Installation

Status of RSpec Tests

BETYdb - BETYdb 4.17

Important bug fixes to Bulk Upload Wizard

Summary of Changes

Bug Fixes

Publication of the beta API as version 1 (`/api/v1`); major speed-up of Bulk Upload wizard and several bug fixes.

Implements API query of sites by geographical coordinates; fixes constraints and sequence numbering of `sites_cultivars` table