Recent Releases of PlanktoScope
PlanktoScope - PlanktoScope OS v2024.0.0
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope software.
About this pre-release
Starting with v2024.0.0, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0 includes major improvements over v2023.9.0, including the complete replacement of the old camera controller which turned out to be buggy in v2023.9.0. Please review the major changes (listed below), try out this release, and formally file bug reports on GitHub Issues or informally let us know about any bugs you encounter in the #6-dev-software
channel on the PlanktoScope Slack workspace! Here are quick links to the SD card images - you should download the image specific to your PlanktoScope hardware version:
-
fairscope-latest
: use this if your PlanktoScope hardware is v2.6. -
planktoscopehat
: use this if your PlanktoScope hardware is v2.5, v2.3, or some other nonstandard design using the PlanktoScope HAT. Note: you can also use this SD card image on the v2.6 PlanktoScope hardware, it'll just have a setup step on the Node-RED dashboard's homepage which you wouldn't need to do if you used thefairscope-latest
image. -
adafruithat
: use this if your PlanktoScope hardware is v2.1 or some other nonstandard design using the Adafruit Stepper HAT.
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module; and to Romain Fauvel (@RomainFauvel) for a code contribution in the segmenter! - Thanks to Thibaut Pollina (@tpollina) and Melissa Djadoun (@melissadjadoun) for documentation contributions!
- Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Guillaume Leguen (@gheleguen), Johan Viljoen, Kevin Du Close (@kevin-duclos), Loïc Vassaux-Artur (@BayouS), Melissa Djadoun (@melissadjadoun), Morgan Coulm (@redhox), Oumayma Elbez (@Oumayma-hy), Satoshi Kitajima (@kitazy52), Patrick White, Pierre Kostyrka (@pkostyrka), Tanguy Cebron, Thibaut Pollina (@tpollina), and Wassim Chakroun (@W7CH), for contributing feature requests, bug reports, and software+documentation testing efforts for this release!
- Special thanks to Fabien and Satoshi for a heroic troubleshooting effort with a tricky, hard-to-reproduce bug involving the new camera driver introduced for v2024.0.0!
Changelog since v2023.9.0
This changelog only lists items we think you are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added
- A
fairscope-latest
SD card image is now built for FairScope customers; its only difference from theplanktoscopehat
SD card image is that the default hardware settings are set for the v2.6 hardware, instead of the v2.5 hardware. - On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
- The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
Changed
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - The first time you boot up the
planktoscopehat
SD card image, the Node-RED dashboard's homepage will ask you to select the hardware version of your PlanktoScope - instead of assuming that you are using the v2.5 hardware (and providing default hardware settings for the v2.5 hardware). - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before; accordingly, the Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting, and the default ISO value has been changed from 100 to 150. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default. WARNING: This changes the data-processing behavior of the segmenter, resulting in slightly different outputs compared to the previous version of the segmenter even for the same input datasets!
- The PlanktoScope OS is now built as a 64-bit OS, instead of a 32-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
- SD card images are now provided as
.img.xz
files instead of.img.gz
files; they will still work with the Raspberry Pi Imager software.
Deprecated
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed after v2024.0.0. Instead, you should use the datasets file browser (which you can access by a link on the PlanktoScope's landing page) for backing up and deleting dataset files on your PlanktoScope.
- Various other non-functional or partially-broken pages and panels in the current Node-RED dashboard will also be removed in the next major release of the PlanktoScope OS. See the deprecations in the v2024.0.0 changelog for details.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in the next major release. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS. After v2024.0.0 we will no longer test whether the installer scripts work on 32-bit Raspberry Pi OS images.
Fixed
- The segmenter's previously incorrect method for filtering segmented objects by size has now been corrected to filter object sizes by filled area rather than bounding box area, and directly using the mesh size as the threshold for equivalent spherical diameter (ESD) instead of calculating a fictional ESD. WARNING: This changes the data-processing behavior of the segmenter, resulting in slightly different outputs compared to the previous version of the segmenter even for the same input datasets!
- The segmenter no longer makes its EcoTaxa exports un-importable by EcoTaxa for datasets with more than ~32,000 objects.
- The segmenter now includes the acquisition ID in the filename of the metadata TSV file included with the EcoTaxa export ZIP archive; this is necessary to allow efficient bulk importing of such ZIP archives into EcoTaxa, which was previously prevented by the use of the same
ecotaxa_export.tsv
filename for all metadata TSV files. - The Node-RED dashboard’s "Sample" page's Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - The Node-RED dashboard should now correctly load the flowcell setting from the
config.json
settings file. - Startup is now faster.
Changelog since v2024.0.0-beta.3
Deprecated
- Various non-functional or partially-broken pages and panels in the current Node-RED dashboard will be removed in the next major release of the PlanktoScope OS. See the deprecations in the v2024.0.0 changelog for details.
Fixed
- The segmenter no longer makes its EcoTaxa exports un-importable by EcoTaxa for datasets with more than ~32,000 objects.
Known Problems
- (Known as of v2024.0.0-beta.2) You may encounter issues with severe lag (latency) in the responsiveness of the camera preview stream. We have occasionally observed this behavior, but it has been difficult for us to reproduce consistently; based on results from the testing we were able to do, we've attempted to minimize occurrence of this behavior with changes in v2024.0.0-beta.3. But we're not sure if it will solve the problem for everyone. If you experience this behavior even after v2024.0.0-beta.2, please let us know!
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Pull Requests
- Update tech ref docs about camera & segmentation functionalities by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/503
- Use merge gatekeeper to avoid running slow CI workflows when not needed by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/504
- Bump mkdocs-redirects from 1.2.1 to 1.2.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/501
- Bump poethepoet from 0.29.0 to 0.30.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/500
- Add an operation guide for network-related administration by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/506
- Replace PDFs of v2 & v3 of Fabien's protocol with a v4 draft by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/509
- Fix incorrect use of img_rank field in EcoTaxa export metadata by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/505
- Clarify documentation of URLs for accessing the software by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/510
- Add an operations guide for upgrading/downgrading/resetting the OS by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/511
- Bump jinja2 from 3.1.4 to 3.1.5 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/512
- Bump version numbers for v2024.0.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/513
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2024.0.0-beta.3...software/v2024.0.0
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli 4 months ago
PlanktoScope - PlanktoScope OS (Sunray) v2024.0.0-beta.3
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope software.
About this pre-release
Starting with v2024.0.0, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0-beta.3 is a pre-release version for everyone to test out. It is a release candidate: v2024.0.0-beta.3 will most likely be promoted to the v2024.0.0 stable release. Please try out this pre-release and file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace! Here are quick links to the SD card images - you should download the image specific to your PlanktoScope hardware version:
-
fairscope-latest
: use this if your PlanktoScope hardware is v2.6. -
planktoscopehat
: use this if your PlanktoScope hardware is v2.5, v2.3, or some other nonstandard design using the PlanktoScope HAT. Note: you can also use this SD card image on the v2.6 PlanktoScope hardware, it'll just have a setup step on the Node-RED dashboard's homepage which you wouldn't need to do if you used thefairscope-latest
image. -
adafruithat
: use this if your PlanktoScope hardware is v2.1 or some other nonstandard design using the Adafruit Stepper HAT.
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module; and to Romain Fauvel (@RomainFauvel) for a code contribution in the segmenter! - Thanks to Thibaut Pollina (@tpollina) and Melissa Djadoun (@melissadjadoun) for documentation contributions!
- Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Guillaume Leguen (@gheleguen), Johan Viljoen, Kevin Du Clos (@kevin-duclos), Loïc Vassaux-Artur (@BiyouS), Melissa Djadoun (@melissadjadoun), Morgan Coulm (@redhox), Oumayma Elbez (@Oumayma-hy), Satoshi Kitajima (@kitazy52), Patrick White, Pierre Kostyrka (@pkostyrka), Tanguy Cebron, Thibaut Pollina (@tpollina), and Wassim Chakroun (@W7CH), for contributing feature requests, bug reports, and software+documentation testing efforts for this release!
- Special thanks to Fabien and Satoshi for a heroic troubleshooting effort with a tricky, hard-to-reproduce bug involving the new camera driver introduced for v2024.0.0!
Changelog since v2023.9.0
This changelog only lists items we think you are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- A
fairscope-latest
SD card image is now built for FairScope customers; its only difference from theplanktoscopehat
SD card image is that the default hardware settings are set for the v2.6 hardware, instead of the v2.5 hardware. - The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
Changed:
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - The first time you boot up the
planktoscopehat
SD card image, the Node-RED dashboard's homepage will ask you to select the hardware version of your PlanktoScope - instead of assuming that you are using the v2.5 hardware (and providing default hardware settings for the v2.5 hardware). - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before; accordingly, the Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default. WARNING: This changes the data-processing behavior of the segmenter, resulting in slightly different outputs compared to the previous version of the segmenter even for the same input datasets!
- The PlanktoScope OS is now built as a 64-bit OS, instead of a 32-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
- SD card images are now provided as
.img.xz
files instead of.img.gz
files; they will still work with the Raspberry Pi Imager software. - The default ISO value is now 150 instead of 100.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed after v2024.0.0. Instead, you should use the datasets file browser (which you can access by a link on the PlanktoScope's landing page) for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about the picamera2-based imaging module's reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS. After v2024.0.0 we may stop testing whether the installer scripts work on 32-bit Raspberry Pi OS images.
Fixed:
- The segmenter's previously incorrect method for filtering segmented objects by size has now been corrected to filter object sizes by filled area rather than bounding box area, and directly using the mesh size as the threshold for equivalent spherical diameter (ESD) instead of calculating a fictional ESD. WARNING: This changes the data-processing behavior of the segmenter, resulting in slightly different outputs compared to the previous version of the segmenter even for the same input datasets!
- The segmenter now includes the acquisition ID in the filename of the metadata TSV file included with the EcoTaxa export ZIP archive; this is necessary to allow efficient bulk importing of such ZIP archives into EcoTaxa, which was previously prevented by the use of the same
ecotaxa_export.tsv
filename for all metadata TSV files. - The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - The Node-RED dashboard should now correctly load the flowcell setting from the
config.json
settings file. - Startup is now faster.
Changelog since v2024.0.0-beta.2
Fixed
- The segmenter's previously incorrect method for filtering segmented objects by size has now been corrected to filter object sizes by filled area rather than bounding box area, and directly using the mesh size as the threshold for equivalent spherical diameter (ESD) instead of calculating a fictional ESD. WARNING: This changes the data-processing behavior of the segmenter, resulting in slightly different outputs compared to the previous version of the segmenter even for the same input datasets!
- A regression in handling of the mDNS domain name
pkscope-{machine-name}.local
(likely introduced by v2024.0.0-beta.0) has been fixed, so that now that type of domain name is usable again.
Known Problems
- (Known as of v2024.0.0-beta.2) You may encounter issues with severe lag (latency) in the responsiveness of the camera preview stream. We have occasionally observed this behavior, but it has been difficult for us to reproduce consistently; based on results from the testing we were able to do, we've attempted to minimize occurrence of this behavior with changes in v2024.0.0-beta.3. But we're not sure if it will solve the problem for everyone. If you experience this behavior even after v2024.0.0-beta.2, please let us know!
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Pull Requests
- Bump mkdocs-git-revision-date-localized-plugin from 1.2.7 to 1.2.9 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/484
- Bump pymdown-extensions from 10.9 to 10.11.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/482
- Bump mkdocs-macros-plugin from 1.0.5 to 1.2.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/481
- Bump poethepoet from 0.28.0 to 0.29.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/483
- fixing the typo on the systemd-timesyncd.service by @redhox in https://github.com/PlanktoScope/PlanktoScope/pull/485
- Bump pymdown-extensions from 10.11.2 to 10.12 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/491
- Bump mkdocs-git-revision-date-localized-plugin from 1.2.9 to 1.3.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/490
- Bump mkdocs-charts-plugin from 0.0.11 to 0.0.12 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/492
- Bump mkdocs-macros-plugin from 1.2.0 to 1.3.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/489
- Add PDF of v3 of Fabien's PlanktoScope operation protocol by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/495
- Update reference docs on segmenter size thresholding by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/496
- Integrate various small-but-consequential fixes for v2024.0.0-beta.3 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/497
- Bump version numbers for v2024.0.0-beta.3 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/498
New Contributors
- @redhox made their first contribution in https://github.com/PlanktoScope/PlanktoScope/pull/485
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2024.0.0-beta.2...software/v2024.0.0-beta.3
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli 5 months ago
PlanktoScope - software/v2024.0.0-beta.2
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope software.
About this pre-release
Starting with v2024.0.0, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0-beta.2 is a pre-release version for everyone to test out. It is a release candidate: if no major software regressions are discovered and reported, v2024.0.0-beta.2 will be promoted to the v2024.0.0 stable release. Please try out this pre-release and file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace! Here are quick links to the SD card images - you should download the image specific to your PlanktoScope hardware version:
-
fairscope-latest
: use this if your PlanktoScope hardware is v2.6. -
planktoscopehat
: use this if your PlanktoScope hardware is v2.5, v2.3, or some other nonstandard design using the PlanktoScope HAT. Note: you can also use this SD card image on the v2.6 PlanktoScope hardware, it'll just have a setup step on the Node-RED dashboard's homepage which you wouldn't need to do if you used thefairscope-latest
image. -
adafruithat
: use this if your PlanktoScope hardware is v2.1 or some other nonstandard design using the Adafruit Stepper HAT.
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module; and to Romain Fauvel (@RomainFauvel) for a code contribution in the segmenter! - Thanks to Thibaut Pollina (@tpollina) and Melissa Djadoun (@melissadjadoun) for documentation contributions!
- Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Guillaume Leguen (@gheleguen), Johan Viljoen, Kevin Du Clos (@kevin-duclos), Loïc Vassaux--Artur (@BiyouS), Melissa Djadoun (@melissadjadoun), Morgan Coulm (@redhox), Oumayma Elbez (@Oumayma-hy), Satoshi Kitajima (@kitazy52), Patrick White, Pierre Kostyrka (@pkostyrka), Tanguy Cebron, Thibaut Pollina (@tpollina), and Wassim Chakroun (@W7CH), for contributing feature requests, bug reports, and software+documentation testing efforts for this release!
Changelog since v2023.9.0
This changelog only lists items we think you are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- A
fairscope-latest
SD card image is now built for FairScope customers; its only difference from theplanktoscopehat
SD card image is that the default hardware settings are set for the v2.6 hardware, instead of the v2.5 hardware. - The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
Changed:
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - The first time you boot up the
planktoscopehat
SD card image, the Node-RED dashboard's homepage will ask you to select the hardware version of your PlanktoScope - instead of assuming that you are using the v2.5 hardware (and providing default hardware settings for the v2.5 hardware). - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before; accordingly, the Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default.
- The PlanktoScope OS is now built as a 64-bit OS, instead of a 32-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
- SD card images are now provided as
.img.xz
files instead of.img.gz
files; they will still work with the Raspberry Pi Imager software. - The default ISO value is now 150 instead of 100.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed in v2024.1.0 (the next release after v2024.0.0). Instead, you should use the datasets file browser (which you can access by a link on the PlanktoScope's landing page) for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about the picamera2-based imaging module's reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS. After v2024.0.0 we may stop testing whether the installer scripts work on 32-bit Raspberry Pi OS images.
Fixed:
- The segmenter now includes the acquisition ID in the filename of the metadata TSV file included with the EcoTaxa export ZIP archive; this is necessary to allow efficient bulk importing of such ZIP archives into EcoTaxa, which was previously prevented by the use of the same
ecotaxa_export.tsv
filename for all metadata TSV files. - The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - The Node-RED dashboard should now correctly load the flowcell setting from the
config.json
settings file. - Startup is now faster.
Changelog since v2024.0.0-beta.1
Changed
- The default ISO value is now 150 instead of 100.
Fixed
- The segmenter now includes the acquisition ID in the filename of the metadata TSV file included with the EcoTaxa export ZIP archive; this is necessary to allow efficient bulk importing of such ZIP archives into EcoTaxa, which was previously prevented by the use of the same
ecotaxa_export.tsv
filename for all metadata TSV files. - The hardware controller's new (libcamera-based) camera controller now initializes its default image gain based on camera sensor type in order to match the GUI's new default ISO value of 150, instead of initializing the image gain to some other default value.
Known Problems
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Pull Requests
- Apply max xz compression on tag-triggered OS build workflows by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/445
- Improve docs related to software setup by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/446
- Try to trigger docs deployment from docs release-channel workflows by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/449
- Import v2.1 hardware setup docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/448
- Bump docker/build-push-action from 5 to 6 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/452
- Bump caddy from 2.8.1 to 2.8.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/451
- Bump ethanjli/cached-download-action from 0.1.1 to 0.1.2 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/453
- Bump poethepoet from 0.26.1 to 0.27.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/450
- Import additional diagrams from planktoscope.org's "Get Your Kit" page by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/454
- Bump certifi from 2024.2.2 to 2024.7.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/456
- Don't activate Forklift filesystem overlays before booted VM setup by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/457
- Bump zipp from 3.18.1 to 3.19.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/458
- Describe default Wi-Fi hotspot password in docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/459
- Improve MQTT API reference docs by @melissadjadoun in https://github.com/PlanktoScope/PlanktoScope/pull/402
- Bump pymdown-extensions from 10.8.1 to 10.9 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/462
- Replace a TODO placeholder in the v2.1 hardware assembly guide by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/463
- Backport/docs setup hardware v2.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/465
- Add a cleanup script to simplify custom SD card image creation by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/466
- Change default ISO to 150 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/467
- Bump mkdocs-charts-plugin from 0.0.10 to 0.0.11 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/469
- Bump mkdocs-markdownextradata-plugin from 0.2.5 to 0.2.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/471
- Bump poethepoet from 0.27.0 to 0.28.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/470
- Bump mkdocs-git-revision-date-localized-plugin from 1.2.6 to 1.2.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/468
- Use the acquisition ID in the filename of the EcoTaxa metadata files by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/476
- Bump version numbers for v2024.0.0-beta.2 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/479
New Contributors
- @melissadjadoun made their first contribution in https://github.com/PlanktoScope/PlanktoScope/pull/402
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2024.0.0-beta.1...software/v2024.0.0-beta.2
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli 7 months ago
PlanktoScope - PlanktoScope OS (Sunray) v2024.0.0-beta.1
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope software.
About this pre-release
Starting with v2024.0.0, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0-beta.1 is a pre-release version for everyone to test out. It may have undiscovered software bugs, breakage, and other problems. If you decide to try out this pre-release, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace! Here are quick links to the SD card images - you should download the image specific to your PlanktoScope hardware version:
-
fairscope-latest
: use this if your PlanktoScope hardware is v2.6. -
planktoscopehat
: use this if your PlanktoScope hardware is v2.5, v2.3, or some other nonstandard design using the PlanktoScope HAT. Note: you can also use this SD card image on the v2.6 PlanktoScope hardware, it'll just have a setup step on the Node-RED dashboard's homepage which you wouldn't need to do if you used thefairscope-latest
image. -
adafruithat
: use this if your PlanktoScope hardware is v2.1 or some other nonstandard design using the Adafruit Stepper HAT.
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module! - Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Kevin Du Clos (@kevin-duclos), Satoshi Kitajima (@kitazy52), Pierre Kostyrka (@pkostyrka), Tanguy Cebron, and Thibaut Pollina (@tpollina), for contributing feature requests, bug reports, and software testing efforts for this release!
Changelog since v2023.9.0
This changelog only lists items we think you are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- A
fairscope-latest
SD card image is now built for FairScope customers; its only difference from theplanktoscopehat
SD card image is that the default hardware settings are set for the v2.6 hardware, instead of the v2.5 hardware. - The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
Changed:
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - The first time you boot up the
planktoscopehat
SD card image, the Node-RED dashboard's homepage will ask you to select the hardware version of your PlanktoScope - instead of assuming that you are using the v2.5 hardware (and providing default hardware settings for the v2.5 hardware). - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before; accordingly, the Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default.
- The PlanktoScope OS is now built as a 64-bit OS, instead of a 32-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
- SD card images are now provided as
.img.xz
files instead of.img.gz
files; they will still work with the Raspberry Pi Imager software.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed in v2024.1.0 (the next release after v2024.0.0). Instead, you should use the datasets file browser (which you can access by a link on the PlanktoScope's landing page) for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about the picamera2-based imaging module's reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS. After v2024.0.0 we may stop testing whether the installer scripts work on 32-bit Raspberry Pi OS images.
Fixed:
- The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - The Node-RED dashboard should now correctly load the flowcell setting from the
config.json
settings file. - Startup is now faster.
Changelog since v2024.0.0-beta.0
Added
- A
fairscope-latest
SD card image is now built for FairScope customers; its only difference from theplanktoscopehat
SD card image is that the default hardware settings are set for the v2.6 hardware, instead of the v2.5 hardware.
Changed
-
The first time you boot up the
planktoscopehat
SD card image, the Node-RED dashboard's homepage will ask you to select the hardware version of your PlanktoScope - instead of assuming that you are using the v2.5 hardware (and providing default hardware settings for the v2.5 hardware). -
SD card images are now provided as
.img.xz
files instead of.img.gz
files; they will still work with the Raspberry Pi Imager software.
Fixed
- The Node-RED dashboard should now correctly load the flowcell setting from the
config.json
settings file.
Known Problems
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Pull Requests
- Refactor GitHub Actions pre-caching of container images by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/427
- Load container images into containerd, without QEMU VM by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/428
- Pin initial pallet for use in factory-reset process by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/429
- Try to fix regression in starting hardware controller by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/431
- Bump urllib3 from 2.2.1 to 2.2.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/433
- Bump ws, engine.io and socket.io-adapter in /software/node-red-dashboard by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/434
- Fix OS builds on PRs from external fork repos by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/435
- Move v2.6 default hardware version to a FairScope-specific OS image by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/430
- Add an automatic timeout to the QEMU VM setup step by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/436
- Un-deprecate the
planktoscope.local
mDNS name by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/438 - Show dialog on homepage when
config.json
's hw version is null by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/432 - Fix loading of default flowcell type into Fluidic Acquisition page by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/439
- Try to move repo checkout to start of OS image build workflow by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/440
- Add CI workflows to fast-forward release branches upon tag push by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/441
- Bump version numbers for v2024.0.0-beta.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/442
- Fix OS image filename generation from pushed release tags by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/443
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2024.0.0-beta.0...software/v2024.0.0-beta.1
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli 10 months ago
PlanktoScope - PlanktoScope OS (Sunray) v2024.0.0-beta.0
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope Operating System (PlanktoScope OS).
About this pre-release
Starting with this beta pre-release, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0-beta.0 is a pre-release version for everyone to test out. It may have undiscovered software bugs, breakage, and other problems. If you decide to try out this pre-release, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module! - Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Kevin Du Clos (@kevin-duclos), Satoshi Kitajima (@kitazy52), Pierre Kostyrka (@pkostyrka), Tanguy Cebron, and Thibaut Pollina (@tpollina), for contributing feature requests, bug reports, and software testing efforts for this release!
Changelog since v2023.9.0
This changelog only lists items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
Changed:
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - On the
planktoscopehat
version of the SD card image, the default hardware configuration has been changed from the v2.5 PlanktoScope hardware to the v2.6 hardware. If you're upgrading the software on a PlanktoScope with v2.5 hardware, you will need to change the hardware configuration back to v2.5 on the "Hardware Settings" page of the Node-RED dashboard. - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before; accordingly, the Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default.
- The PlanktoScope OS is now built as a 64-bit OS, instead of a 32-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed in v2024.1.0 (the next release after v2024.0.0). Instead, you should use the datasets file browser (which you can access by a link on the PlanktoScope's landing page) for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about the picamera2-based imaging module's reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS. After v2024.0.0 we may stop testing whether the installer scripts work on 32-bit Raspberry Pi OS images.
Fixed:
- The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - Startup is now faster.
Changelog since v2024.0.0-alpha.2
Changed
-
The PlanktoScope OS is now built as a 64-bit OS. This change speeds up the segmenter, potentially by a factor of 2.
-
The Node-RED dashboard's ISO selector has been changed to a slider to enable finer adjustment of the ISO setting.
Deprecated
- Now that the PlanktoScope project is building 64-bit versions of the PlanktoScope OS, 32-bit versions of the PlanktoScope OS are no longer built or officially supported by the project. However, for v2024.0.0 you can still use the installer scripts (with the PlanktoScope OS's non-standard installation process) to set up the PlanktoScope OS on 32-bit base images of the Raspberry Pi OS.
Fixed
- Because v2024.0.0-alpha.2 had fixed an incorrect formula for converting the ISO setting to an image gain for the camera hardware, in that pre-release it was possible (in the planktoscopehat build of the PlanktoScope OS) to use the Node-RED dashboard to attempt to set an ISO value (anything above 650) which would be rejected and ignored by the Python hardware controller because it resulted in an image gain value above the maximum allowed value. Now the planktoscopehat version of the Node-RED dashboard limits the ISO setting to 650.
Known Problems
- (Known as of v2024.0.0-beta.0) In some cases, changing the ISO setting may cause the camera preview to disappear. You can reload the camera preview by refreshing your web browser tab. We will fix this problem before the v2024.0.0 stable release.
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
What's Changed
- build(deps): bump poethepoet from 0.25.0 to 0.26.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/406
- build(deps): bump mkdocs-material from 9.5.17 to 9.5.18 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/403
- build(deps): bump pymdown-extensions from 10.7.1 to 10.8.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/405
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.4 to 1.2.5 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/404
- build(deps): bump jinja2 from 3.1.3 to 3.1.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/407
- Remove FAQ item about PlanktoScope cost by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/409
- Bump mkdocs-material to fix CI build by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/410
- Deploy
machine-name
via Forklift with pallet-standard rather than the OS setup scripts by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/411 - Try to fix
docker/build-push-action
in PRs from external forks by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/413 - build(deps): bump requests from 2.31.0 to 2.32.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/414
- Bump mkdocs-git-revision-date-localized-plugin from 1.2.5 to 1.2.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/417
- Bump caddy from 2.7.6 to 2.8.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/420
- Bump pygments from 2.17.2 to 2.18.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/419
- Bump mkdocs-glightbox from 0.3.7 to 0.4.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/418
- Start building 64-bit OS images in GitHub Actions by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/416
- Try to fix installer versioning in CI OS builds outside PRs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/421
- Bump version numbers for v2024.0.0-beta.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/423
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2024.0.0-alpha.2...software/v2024.0.0-beta.0
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli 11 months ago
PlanktoScope - PlanktoScope OS (Sunray) v2024.0.0-alpha.2
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope Operating System (PlanktoScope OS).
About this pre-release
Starting with this pre-release, we are calling the software we provide in our SD card images the "PlanktoScope OS" (because it is a an operating system for the PlanktoScope).
PlanktoScope OS v2024.0.0-alpha.2 is a pre-release version for developers to test out. It may have undiscovered software bugs, breakage, and other problems. If you decide to try out this pre-release, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Contributors
The PlanktoScope OS is a community effort, and the v2024.0.0 release would not have been possible without the PlanktoScope community:
- Thanks to Wassim Chakroun (@W7CH) for code contributions in the new
picamera2
-based image-acquisition module! - Thanks to (in alphabetical order) Ana Fernandez Carrera, Adam Larson (@babo989), Fabien Lombard (@fabienlombard), Kevin Du Clos (@kevin-duclos), Satoshi Kitajima (@kitazy52), Pierre Kostyrka (@pkostyrka), Tanguy Cebron, and Thibaut Pollina (@tpollina), for contributing feature requests, bug reports, and software testing efforts for this release!
Changelog since v2023.9.0
This changelog only lists items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- The PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port (and a USB-to-Ethernet adapter, if one is attached) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page will now prompt you to reset the PlanktoScope's system clock to match the time from your web browser, if the two times are very different. This enables you to temporarily fix an inaccurate system clock on your PlanktoScope, at least until the next time you unplug the PlanktoScope from power.
Changed:
- The
pscopehat
version of the SD card image has been renamed toplanktoscopehat
for more consistent naming. - On the
planktoscopehat
version of the SD card image, the default hardware configuration has been changed from the v2.5 PlanktoScope hardware to the v2.6 hardware. If you're upgrading the software on a PlanktoScope with v2.5 hardware, you will need to change the hardware configuration back to v2.5 on the "Hardware Settings" page of the Node-RED dashboard. - The hardware controller's camera driver has been fully rewritten to fix various problems we had with
raspimjpeg
(which has now been replaced with a different camera control mechanism). This change may require you to use different ISO and white balance gains than before. - Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
- The "USB backup" functionality of the Node-RED dashboard will be removed in v2024.1.0 (the next release after v2024.0.0). Instead, you should use the datasets file browser for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about its reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
Fixed:
- The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - Startup is now faster.
Changelog since v2024.0.0-alpha.1
Changed
-
Previously, the segmenter's default behavior was to subtract consecutive masks to try to mitigate image-processing issues with objects which get stuck to the flowcell during imaging. However, when different objects occupied the same space in consecutive frames, the subtraction behavior would subtract one object's mask from the mask of the other object in the following frame, which would produce clearly incorrect masks. This behavior is no longer enabled by default.
-
The image quality of frames in the camera preview stream is increased, and frames also have greater width and height (960x720); previously, it was 640x480.
-
The default settings configuration file for the
planktoscopehat
SD card image is now for the v2.6 PlanktoScope hardware; previously, it was still for the v2.5 hardware.
Deprecated
- The "USB backup" functionality of the Node-RED dashboard will be removed in v2024.1.0 (the next release after v2024.0.0). Instead, you should use the datasets file browser for backing up and deleting dataset files on your PlanktoScope.
- The
raspimjpeg
-based imaging module in the Python hardware controller has not yet been deleted, so that you can change the Python hardware controller code to switch back from the new picamera2-based imaging module if picamera2 ends up causing big problems for you. However, we are deprecating theraspimjpeg
-based imaging module, and we will fully delete it in a future release after we get sufficient information about its reliability for PlanktoScope users. - Portainer will no longer be installed/provided by default after v2024.0.0, because it takes up lots of disk space and it is difficult to set up and its functionality overlaps with the much simpler Dozzle. After we remove Portainer from the default SD card image, you can use Forklift to install Portainer.
Fixed
- The incorrect scaling factor for converting between ISO settings (in the Node-RED dashboard) and camera image gains in the new picamera2-based imager is fixed. This should cause images (and the camera preview) to appear brighter than v2024.0.0-alpha.1 when the same ISO settings are used.
Known Problems
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. We are working on a solution to this problem, for a future release after v2024.0.0.
Planning, management, and maintenance of the v2024.0.0 release of PlanktoScope OS is led by Ethan Li (@ethanjli), with input and oversight from attendees of the weekly PlanktoScope software development meetings. If this release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software channel on the PlanktoScope Slack workspace!
Pull Requests
- Try to fix GitHub Actions permissions for docker/build-and-push action by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/338
- build(deps): bump caddy from 2.7.5 to 2.7.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/337
- build(deps): bump actions/configure-pages from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/331
- build(deps): bump mkdocs-material from 9.5.1 to 9.5.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/336
- build(deps): bump actions/deploy-pages from 2 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/329
- build(deps): bump actions/upload-pages-artifact from 2 to 3 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/330
- build(deps): bump actions/setup-python from 4 to 5 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/328
- build(deps): bump actions/upload-artifact from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/327
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.1 to 1.2.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/332
- build(deps): bump mkdocs-glightbox from 0.3.5 to 0.3.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/335
- build(deps): bump pymdown-extensions from 10.5 to 10.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/334
- Fix issue with required status checks not running by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/339
- build(deps): bump mkdocs-minify-plugin from 0.7.1 to 0.7.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/333
- Fix some issues with running the distro setup scripts on RPi OS 12 (bookworm) by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/324
- Allow running Docker commands without sudo by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/341
- Bump pallet-standard to add Dozzle by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/342
- Update hostname separately from the
/etc/hosts
file by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/344 - build(deps): bump gitpython from 3.1.40 to 3.1.41 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/345
- build(deps): bump jinja2 from 3.1.2 to 3.1.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/347
- Add Prometheus+Grafana-based monitoring of host machine, enable resetting system time to browser time by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/346
- Run
forklift plt apply
with parallelization by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/348 - os/pallet: Move some scripts and local files over into management by forklift by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/350
- Add support for
usb0
,eth1
, andwlan1
interfaces by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/351 - Deploy the backend segmenter with forklift rather than the distro setup scripts by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/352
- Replace
<object>
tags with<iframe>
tags by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/361 - Don't suppress wget errors in the docs's non-standard install commands by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/362
- Replace slider with text input to set pump volume by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/363
- Export the filtered volume metadata field on all sample types by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/364
- Repopulate input fields of input forms on Sample page by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/365
- Bump mkdocs-material to bump pillow to resolve dependabot alert by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/366
- build(deps): bump mkdocs-glightbox from 0.3.6 to 0.3.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/367
- build(deps): bump mkdocs-minify-plugin from 0.7.2 to 0.8.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/368
- Bump changelog version number for v2024.0.0-alpha.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/369
- Bump Python segmenter by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/372
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.2 to 1.2.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/373
- build(deps): bump mkdocs-material from 9.5.6 to 9.5.12 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/375
- build(deps): bump poethepoet from 0.24.4 to 0.25.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/374
- Move tech specs in the docs to their own pages by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/379
- Switch from raspimjpeg to picamera2 for image acquisition by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/380
- Bump forklift from v0.5.3 to v0.6.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/381
- Fix the Node-RED dashboard's "Restart segmenter" button by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/382
- Bump version numbers for v2024.0.0-alpha.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/383
- Make small improvements to the software install docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/386
- Fix various small issues in the Node-RED dashboard by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/385
- Temporarily disable MkDocs social preview cards by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/387
- Add technical reference documentation by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/390
- Add operating systems-oriented technical reference documentation by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/391
- Try to re-enable MkDocs social preview cards by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/392
- build(deps): bump actions/configure-pages from 4 to 5 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/393
- build(deps): bump pymdown-extensions from 10.7 to 10.7.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/394
- build(deps-dev): bump pillow from 10.2.0 to 10.3.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/395
- Fix papercuts in the setup instructions for the docs dev environment by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/396
- Migrate OS config files into the Forklift pallet by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/397
- Update software docs for FS overlays, and for renaming to "PlanktoScope OS" by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/398
- Fix various blockers for v2024.0.0-beta.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/399
- Bump version numbers for v2024.0.0-alpha.2 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/400
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0...software/v2024.0.0-alpha.2
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli about 1 year ago
PlanktoScope - Software: Sunray v2024.0.0-alpha.1
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes major fixes for various problems and improvements to existing functionality with the PlanktoScope software.
About this pre-release
Software v2024.0.0-alpha.1 is a pre-release version for developers to test out. It probably has undiscovered software bugs, breakage, and other problems. If you decide to try out this pre-release, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Contributors
The PlanktoScope software distro is a community effort, and this release would not have been possible without the PlanktoScope community:
- Thanks to @W7CH for code contributions in the new
picamera2
-based image-acquisition module! - Thanks to (in alphabetical order) Ana Fernandez Carrera, @babo989, @fabienlombard, @kevin-duclos, @kitazy52, @pkostyrka, and @tpollina, for contributing feature requests, bug reports, and software testing efforts for this release!
Changelog since v2024.0.0-alpha.0
This changelog only lists items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Changed:
- The hardware controller now uses
picamera2
instead ofraspimjpeg
for camera control; this fixes various issues we had withraspimjpeg
. This change may require different ISO and white balance gains to be used. The camera preview stream may also be displayed more smoothly.
Deprecated:
- The current Node-RED dashboard (both the version for the Adafruit HAT and the version for the PlanktoScope HAT) is transitioning to maintenance mode: no new features will be added, and any bugs will be only be fixed if someone volunteers to fix them. The current Node-RED dashboard will be completely replaced by a fully-rewritten Node-RED dashboard, though there is no timeline for completion of that new dashboard. Currently, our plan for deprecating and eventually removing the current Node-RED dashboard is as follows: maintenance mode (no new features, only some bugfixes), then deprecation (no maintenance; not enabled by default, but still installed), then removal (not installed by default, but anyone is free to install it and see if it still works); deprecation will not occur before the rewritten Node-RED dashboard is stable for general-purpose usage. If you have concerns, please share your feedback on GitHub or on the PlanktoScope Slack.
Known Problems
(known problems are not catalogued in alpha pre-releases)
Planning, management, and maintenance of the v2024.0.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from attendees of the weekly PlanktoScope software development meetings. If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software channel on the PlanktoScope Slack workspace!
Pull Requests
- Try to fix GitHub Actions permissions for docker/build-and-push action by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/338
- build(deps): bump caddy from 2.7.5 to 2.7.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/337
- build(deps): bump actions/configure-pages from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/331
- build(deps): bump mkdocs-material from 9.5.1 to 9.5.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/336
- build(deps): bump actions/deploy-pages from 2 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/329
- build(deps): bump actions/upload-pages-artifact from 2 to 3 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/330
- build(deps): bump actions/setup-python from 4 to 5 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/328
- build(deps): bump actions/upload-artifact from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/327
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.1 to 1.2.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/332
- build(deps): bump mkdocs-glightbox from 0.3.5 to 0.3.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/335
- build(deps): bump pymdown-extensions from 10.5 to 10.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/334
- Fix issue with required status checks not running by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/339
- build(deps): bump mkdocs-minify-plugin from 0.7.1 to 0.7.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/333
- Fix some issues with running the distro setup scripts on RPi OS 12 (bookworm) by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/324
- Allow running Docker commands without sudo by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/341
- Bump pallet-standard to add Dozzle by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/342
- Update hostname separately from the
/etc/hosts
file by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/344 - build(deps): bump gitpython from 3.1.40 to 3.1.41 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/345
- build(deps): bump jinja2 from 3.1.2 to 3.1.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/347
- Add Prometheus+Grafana-based monitoring of host machine, enable resetting system time to browser time by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/346
- Run
forklift plt apply
with parallelization by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/348 - os/pallet: Move some scripts and local files over into management by forklift by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/350
- Add support for
usb0
,eth1
, andwlan1
interfaces by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/351 - Deploy the backend segmenter with forklift rather than the distro setup scripts by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/352
- Replace
<object>
tags with<iframe>
tags by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/361 - Don't suppress wget errors in the docs's non-standard install commands by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/362
- Replace slider with text input to set pump volume by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/363
- Export the filtered volume metadata field on all sample types by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/364
- Repopulate input fields of input forms on Sample page by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/365
- Bump mkdocs-material to bump pillow to resolve dependabot alert by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/366
- build(deps): bump mkdocs-glightbox from 0.3.6 to 0.3.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/367
- build(deps): bump mkdocs-minify-plugin from 0.7.2 to 0.8.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/368
- Bump changelog version number for v2024.0.0-alpha.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/369
- Bump Python segmenter by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/372
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.2 to 1.2.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/373
- build(deps): bump mkdocs-material from 9.5.6 to 9.5.12 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/375
- build(deps): bump poethepoet from 0.24.4 to 0.25.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/374
- Move tech specs in the docs to their own pages by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/379
- Switch from raspimjpeg to picamera2 for image acquisition by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/380
- Bump forklift from v0.5.3 to v0.6.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/381
- Fix the Node-RED dashboard's "Restart segmenter" button by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/382
- Bump version numbers for v2024.0.0-alpha.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/383
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0...software/v2024.0.0-alpha.1
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli about 1 year ago
PlanktoScope - Software: Sunray v2024.0.0-alpha.0
The v2024.0.0 software release, which is part of the "Sunray" series of software releases started with v2023.9.0, includes fixes for various problems and improvements to existing functionality with the PlanktoScope software. Significant behind-the-scenes changes are also included in this release.
About this pre-release
Software v2024.0.0-alpha.0 is a pre-release version for developers to test out. It definitely has undiscovered software bugs, breakage, and other problems. If you decide to try out this pre-release, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Changelog
This changelog only lists items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Added:
- Added network configuration support for a few additional network interfaces (USB tethering, USB Wi-Fi dongle, and USB-to-Ethernet adapter). Thus, the PlanktoScope now shares internet access over its Wi-Fi hotspot and Ethernet port(s) from a phone connected to it in USB tethering mode.
- On the Node-RED dashboard, the System Monitoring page now shows a prompt for the user to reset the PlanktoScope’s system clock to match the time from their web browser, if the two clocks differ by more than one minute.
Changed:
- In the Node-RED dashboard, various input elements have been improved for better usability.
- On the Node-RED dashboard, the System Monitoring page’s monitoring panel has been replaced with an embedded Grafana dashboard, with additional graphs available in an expandable subpanel. The Grafana dashboard can be customized by advanced users.
Fixed:
- The Node-RED dashboard’s “Sample” page’s Filtered Volume field is now exported as the
sample_total_volume
metadata field for all sample types; previously it was only exported for horizontal plankton tow samples. - Startup is now faster.
Known Problems
(known problems are not catalogued in alpha pre-releases)
Pull Requests
- Try to fix GitHub Actions permissions for docker/build-and-push action by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/338
- build(deps): bump caddy from 2.7.5 to 2.7.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/337
- build(deps): bump actions/configure-pages from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/331
- build(deps): bump mkdocs-material from 9.5.1 to 9.5.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/336
- build(deps): bump actions/deploy-pages from 2 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/329
- build(deps): bump actions/upload-pages-artifact from 2 to 3 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/330
- build(deps): bump actions/setup-python from 4 to 5 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/328
- build(deps): bump actions/upload-artifact from 3 to 4 by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/327
- build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.1 to 1.2.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/332
- build(deps): bump mkdocs-glightbox from 0.3.5 to 0.3.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/335
- build(deps): bump pymdown-extensions from 10.5 to 10.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/334
- Fix issue with required status checks not running by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/339
- build(deps): bump mkdocs-minify-plugin from 0.7.1 to 0.7.2 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/333
- Fix some issues with running the distro setup scripts on RPi OS 12 (bookworm) by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/324
- Allow running Docker commands without sudo by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/341
- Bump pallet-standard to add Dozzle by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/342
- Update hostname separately from the
/etc/hosts
file by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/344 - build(deps): bump gitpython from 3.1.40 to 3.1.41 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/345
- build(deps): bump jinja2 from 3.1.2 to 3.1.3 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/347
- Add Prometheus+Grafana-based monitoring of host machine, enable resetting system time to browser time by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/346
- Run
forklift plt apply
with parallelization by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/348 - os/pallet: Move some scripts and local files over into management by forklift by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/350
- Add support for
usb0
,eth1
, andwlan1
interfaces by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/351 - Deploy the backend segmenter with forklift rather than the distro setup scripts by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/352
- Replace
<object>
tags with<iframe>
tags by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/361 - Don't suppress wget errors in the docs's non-standard install commands by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/362
- Replace slider with text input to set pump volume by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/363
- Export the filtered volume metadata field on all sample types by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/364
- Repopulate input fields of input forms on Sample page by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/365
- Bump mkdocs-material to bump pillow to resolve dependabot alert by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/366
- build(deps): bump mkdocs-glightbox from 0.3.6 to 0.3.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/367
- build(deps): bump mkdocs-minify-plugin from 0.7.2 to 0.8.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/368
- Bump changelog version number for v2024.0.0-alpha.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/369
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0...software/v2024.0.0-alpha.0
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli about 1 year ago
PlanktoScope - Software: Sunray v2023.9.0
v2023.9.0 is our first software update in two years, and we’re excited to make it simpler and easier for you to use your PlanktoScope in this update and in future updates!
This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users. This release includes big changes to how you access the PlanktoScope's software.
With this release, we are decoupling the evolution and versioning of the PlanktoScope software from the PlanktoScope hardware. Our goal is to make the latest version of the PlanktoScope software be compatible with every previous supported version of the PlanktoScope hardware. Thus, you may see references to “v2.1 hardware”, “v2.3 hardware”, and “v2.5 hardware” at the same time as “v2023.9.0 software” or “v2.3 software”.
A few notes:
- Anyone trying to build or set up a new Adafruit Stepper HAT-based PlanktoScope should use the v2023.9.0 software instead of the v2.3 software, as v2023.9.0 solves some critical firmware/driver version incompatibilities which Adafruit has caused over the past year on their Adafruit Stepper HATs; those incompatibilities make it difficult to get the v2.3 SD card image to work with Adafruit Stepper HATs purchased since around mid-2022.
- Because of the nature of a known problem related to the camera control system (see the "Known Problems" section for details), we recommend that anyone who's currently relying on the v2.3 software as part of mission-critical operations should probably continue using the v2.3 software for now.
Download links:
- SD card image for PlanktoScopes using the custom PlanktoScope HAT (e.g. v2.3 hardware and above)
- SD card image for PlanktoScopes using the Adafruit Stepper HAT (e.g. v2.1 hardware)
Contributors
The PlanktoScope software distro is a community effort, and this release would not have been possible without the PlanktoScope community:
- Thanks to (in alphabetical order) Ana Fernandez Carrera, Arief Rachman, @fabienlombard, @glx314, @hankertrix, @kitazy52, @pkostyrka, Rodrigo Gonçalves, Savannah Mapes, and @tpollina, for testing out pre-release versions of this software release and reporting bugs!
- Thanks also to @sourceindex for revamping the project documentation which is now included in this software release; and to @LaurentPV for contributing software improvements as part of this software release.
Notable changes since v2.3
Highlights:
- To use the PlanktoScope from your web browser, now you should just open http://pkscope.local/ or (if that doesn’t work) http://home.pkscope/ or (if that doesn’t work either) http://192.168.4.1/, which will show you a landing page with a link to the Node-RED dashboard as well as other useful links. The previous URLs used for accessing the PlanktoScope's Node-RED dashboard (e.g. http://192.168.4.1:1880/ui) no longer work.
- The name of your PlanktoScope (and its Wi-Fi hotspot) has changed to a new naming scheme.
- PlanktoScopes will no longer stay connected to any Wi-Fi network which doesn’t provide internet access.
- The PlanktoScope now acts like a router, so if it has internet access then it will share that internet access with your computer/phone.
- The PlanktoScope project has a new and improved documentation website, at https://docs.planktoscope.community/ .
Added:
- The PlanktoScope can now be accessed from your web browser at http://pkscope.local/ (assuming your web browser supports mDNS), and http://home.pkscope/ (assuming your web browser isn’t configured to use a custom DNS server).
- The PlanktoScope now shares any internet access it has from Wi-Fi with all devices connected to its Ethernet port; and it also shares any internet access it has from Ethernet with all devices connected to its Wi-Fi hotspot.
- An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page. The online version of the documentation is now at https://docs.planktoscope.community/ .
Changed:
- The Node-RED dashboard (previously available at http://planktoscope.local:1880/ui) has been moved to a different URL, accessible by a link on the new landing page (see the note in the “Added” section about new URLs and the landing page).
- The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, and download folders as ZIP files.
- Previously, PlanktoScope machine names were generated as gibberish words like "Babaxio-Detuiau". However, the machine names created by this naming scheme were often difficult to pronounce, remember, and type for people in various languages, and the naming scheme sometimes generated names which sounded like curses or insults in some languages. Now, PlanktoScope machine names are generated as a combination of two words and a number up to five digits long; words are selected from pre-built lists in a language which can be chosen based on localization settings. Currently, word lists are only provided in US English, resulting in names like "metal-slope-23501", "conscious-pocket-1684", and "plant-range-10581"; however, word lists can be added for other languages in the future.
- Previously, the PlanktoScope would stay connected to a Wi-Fi network even if it didn’t have internet access on that network. Now, if it connects to a Wi-Fi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to creating its own Wi-Fi hotspot for your computer or phone to connect to.
Deprecated:
- The
planktoscope.local
mDNS name is no longer recommended for use; instead, you should usepkscope.local
or the machine-specific mDNS name of formatpkscope-{machine-name}.local
. - We might need to eventually transition the adafruithat version of the Node-RED dashboard to maintenance mode (i.e. no new features), if we can't merge it into the pscopehat version of the Node-RED dashboard. Nothing has been decided yet.
Fixed:
- When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value will now be reset, instead of causing the PlanktoScope software to crash after every subsequent restart.
- In certain network configurations, the Node-RED dashboard was used incorrect URLs to embed the MJPEG streams of the camera preview, resulting in a potentially misleading error message being shown about Python scripts. Now, the Node-RED dashboard uses the correct URLs for the MJPEG streams.
- The brightness of the illumination LED for the pscopehat version of software has been reduced; this is a temporary workaround to a bug with our camera control system where saved images are overexposed on the pscopehat version of the hardware even on the default brightness settings with minimum shutter speed and ISO, despite the brightness of the camera preview looking reasonable.
- The
adafruit-blinka
,adafruit-platformdetect
, andadafruit-circuitpython-busdevice
driver libraries are now updated, so that the PlanktoScope software should no longer crash just because it is being used with recent versions of the Adafruit Stepper HAT.
Notable changes since v2023.9.0-beta.2
(this release involves no changes from the v2023.9.0-beta.2 software prerelease; it's just a promotion of v2023.9.0-beta.2 to a stable release)
Known Problems
If you experience any of the following problems, please report it and describe your testing circumstances to help us understand and fix these problems:
- Sometimes, image acquisition will fail to start, with a “camera timeout” error; or sometimes the camera will not respond to camera settings changes. This problem may go away after one or more attempts to restart the Python hardware controller (from the Node-RED dashboard’s “Administration” page), but it might come back sometimes. This is a complicated issue and we still don’t know exactly why it happens.
- You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you file a bug report about this problem, please describe your GPS situation and the time you tried to change the system clock to.
- In certain cases, some computers connected both to a Wi-Fi network providing internet access and to a PlanktoScope may still be unable to access anything on the internet. If you file a bug report about this problem, please describe your operating system and your network configuration for internet access.
- If you restart the Node-RED dashboard without rebooting, you will also need to manually restart the Python hardware controller afterwards, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope.
Planning, management, and maintenance of the v2023.9.0 software release was led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina (@tpollina). If this software release causes new problems for how you are using your PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli over 1 year ago
PlanktoScope - Software: Sunray v2023.9.0-beta.2
v2023.9.0 will be our first software update in around two years, and we’re excited to make it easier for you to use your PlanktoScope in this update and in future updates! The v2023.9.0 software release focuses on improving the software infrastructure of the PlanktoScope software distro in preparation for future software improvements and maintenance, and making various usability improvements.
This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users.
Planning, management, and maintenance of the v2023.9.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina (@tpollina). If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
About this pre-release
Software v2023.9.0-beta.2 is a pre-release testing version for you to test out. It definitely has software bugs and other problems (the problems we know about are listed under the “Known Issues” section near the end of these release notes), so you should not use it for collecting scientific data! We would greatly appreciate it if you could flash the v2023.9.0-beta.2 image to an SD card and let us know about any bugs or problems you encounter while trying to use your PlanktoScope. This will help us to deliver a higher-quality software release with our v2023.9.0 release.
If you decide to try out the v2023.9.0-beta.2 software, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Because this pre-release includes many changes, the changelogs below only list items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Changelog since v2.3
Highlights:
- To use the PlanktoScope from your web browser, now you should just open http://home.pkscope/ or http://pkscope.local/ or http://192.168.4.1/.
- You no longer need to use FileZilla to download and delete the datasets on your PlanktoScope - now you can do it from your web browser.
- The PlanktoScope now acts like a router, so if it has internet access then it will share that internet access with your computer/phone.
PlanktoScopes will no longer stay connected to any Wi-Fi network which doesn’t provide internet access. - PlanktoScopes now use a new naming scheme for machine names.
- We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
- The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
- Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.
Added:
- The PlanktoScope can now be accessed from your web browser at http://home.pkscope/ (assuming your web browser isn’t configured to use a custom DNS server such as
1.1.1.1
or8.8.8.8
) and http://pkscope.local/ (assuming your web browser supports mDNS), in addition to the other URLs which previously worked (http://planktoscope.local/, http://192.168.4.1/, and http://192.168.5.1/).- Each PlanktoScope can now also be accessed at machine-specific URLs of format
http://<machine-name>.pkscope
andhttp://pkscope-<machine-name>.local
, where<machine-name>
is replaced with your PlanktoScope’s machine name (see the note in the “Changed” section below about machine names). - If your web browser tries to do a Google search for the URL instead of actually opening it as a webpage, make sure you type
http://
at the start or/
at the end (e.g. home.pkscope/ or http://home.pkscope/ instead ofhome.pkscope
) - If you connect directly from your computer/phone to the PlanktoScope and then use your web browser to open http://home.pkscope/,
http://<machine-name>.pkscope
,http://pkscope-<machine-name>.local
, http://pkscope.local/, http://192.168.4.1/, or http://192.168.5.1/, you will now see a landing page with a list of useful links to software running on the PlanktoScope.
- Each PlanktoScope can now also be accessed at machine-specific URLs of format
- The PlanktoScope now shares any internet access it has from Wi-Fi with all devices connected to its Ethernet port; and it also shares any internet access it has from Ethernet with all devices connected to its Wi-Fi hotspot, if it’s operating in wireless AP mode.
- An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page. The online version of the documentation is now at https://docs.planktoscope.community .
- The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2 . You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.
- The PlanktoScope documentation pages have been redone in a new system, and improved in various ways. The hardware documentation now describes the v2.5 PlanktoScope hardware. Thanks to Sebastian Wendel (@sourceindex) for starting this work, and for contributing lots of documentation!
Changed:
- The Node-RED dashboard (previously available at http://planktoscope.local:1880/ui) has been moved to a different URL, accessible by a link on the new landing page (see the note in the “Added” section about new URLs and the landing page).
- The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, edit text files, download folders as ZIP files, rename folders, and perform other common tasks.
- This way, you no longer have to install FileZilla in order to copy or delete any of your datasets!
- Previously, PlanktoScope machine names were generated as gibberish words like "Babaxio-Detuiau", and the machine names were used as the names of the Wi-Fi hotspot networks made by the PlanktoScope. However, the machine names created by this naming scheme were often difficult to pronounce, remember, and type for people in various languages, and the naming scheme sometimes generated names which sounded like curses or insults in some languages. Now, PlanktoScope machine names are generated as a combination of two words and a number up to five digits long; words are selected from pre-built lists in a language which can be chosen based on localization settings. Currently, word lists are only provided in US English, resulting in names like "metal-slope-23501", "conscious-pocket-1684", and "plant-range-10581"; however, word lists can be added for other languages in the future, and a user interface will eventually be provided for changing localization settings.
- This change makes it easier for you to tell someone else which PlanktoScope to connect to if multiple PlanktoScope Wi-Fi hotspot networks are available nearby.
- If you know a language besides US English and you’d like to help us add support for generating machine names in that language, please start a discussion on Github or in the
#6-dev-software
channel on the PlanktoScope Slack workspace!
- Previously, the PlanktoScope would stay connected to a wifi network even if it didn’t have internet access on that network. Now, if it connects to a Wi-Fi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to creating its own Wi-Fi hotspot for your computer or phone to connect to.
Deprecated:
- The
planktoscope.local
mDNS name is no longer recommended for use; instead, you should usepkscope.local
or the machine-specific mDNS name of formatpkscope-{machine-name}.local
.- However, we will continue to support
planktoscope.local
for a while, to maintain compatibility with earlier software versions.
- However, we will continue to support
- In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features.
- Depending on the amount of work required, we might be able to make the Node-RED dashboard for the custom PlanktoScope HAT be compatible with the Adafruit HAT. However, we do not have any specific plans yet.
- Regardless, we do plan to continue to fix bugs in the Node-RED dashboard for the Adafruit HAT for some time, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.
Fixed:
- When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file). Previously, invalid white balance values were written to the hardware.json configuration file, which would cause the PlanktoScope software to crash after every subsequent restart until the hardware.json file was manually removed or fixed.
- Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
- In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to show the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
- Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!
- The
adafruit-blinka
andadafruit-platformdetect
software dependencies are now updated to their latest version. As a result, the PlanktoScope software should not crash just because it is being used with recent versions of the Adafruit Stepper HAT.- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
pip3 install --upgrade adafruit-blinka adafruit-platformdetect
command before your PlanktoScope can work properly.
- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
- The default brightness of the illumination LED for the pscopehat version of software has been reduced; this is a temporary workaround to a bug with raspimjpeg where saved images are overexposed even on the default brightness settings with minimum shutter speed and ISO, despite the brightness of raspimjpeg's camera preview looking reasonable.
- Thanks to Laurent Paul Vallet (@LaurentPV) for contributing this fix!
Changelog since v2023.9.0-beta.1
Added:
- A hardware configuration file for PlanktoScope hardware v2.6, which was previously missing, has been added. It is now the default hardware configuration for the pscopehat build of the PlanktoScope SD card image; preset configuration files for hardware v2.3 and v2.5 can be selected instead from the Node-RED dashboard’s “Hardware Settings” page.
Fixed:
- The Node-RED dashboard is now less eager in validating and undoing any changes you make to the white balance which could cause an invalid white balance value. Now, changes you make to the white balance are only validated and applied after you click away from the input field.
- Thanks to Pierre Kostyrka (@pkostyrka) for making a bug report about this issue!
- The default brightness of the illumination LED for the pscopehat version of software has been reduced; this is a temporary workaround to a bug with raspimjpeg where saved images are overexposed even on the default brightness settings with minimum shutter speed and ISO, despite the brightness of raspimjpeg's camera preview looking reasonable.
- Thanks to Laurent Paul Vallet (@LaurentPV) for contributing this fix!
- Now the PlanktoScope should finish booting more quickly the first time it is booted after the SD card is flashed - now the first boot should be roughly as fast as subsequent boots.
Known Problems:
- (Known as of v2023.9.0-beta.0) Sometimes, image acquisition will fail to start, with a “camera timeout” error; or sometimes the camera will not respond to camera settings changes from Node-RED. This problem may go away after one or more reboots, or it may come back after one or more reboots. This is a complicated issue and we still don’t know exactly why it happens. If you experience this problem, please report it and your testing circumstances (e.g. when would the error occur, how repeatable was it, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.1) You may encounter weird behavior with the system time if you try to use Cockpit to manually adjust the system clock. If you experience this problem, please report it and your testing circumstances (e.g. your GPS situation, and the time you tried to change the system clock to), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. The solution will take some work and will not be delivered in the v2023.9.0 stable release.
Pull Requests
- Automatically pull software changelog into the docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/232
- Make Node-RED trigger white-balance input validation less aggressively by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/238
- Update notes in docs about Android's mDNS support by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/246
- build(deps-dev): bump pillow from 10.0.0 to 10.0.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/241
- build(deps-dev): bump linkchecker from 10.2.1 to 10.3.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/243
- build(deps-dev): bump poethepoet from 0.22.1 to 0.24.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/247
- build(deps-dev): bump urllib3 from 2.0.4 to 2.0.6 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/248
- build(deps-dev): bump urllib3 from 2.0.6 to 2.0.7 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/251
- Apply the forklift pallet as part of the SD card setup process by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/254
- Make layout of fields consistent between Node-RED dashboard versions by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/255
- Improve the documentation's software setup instructions by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/256
- Add hardware v2.6 support by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/271
- Add explanatory comment about image width/height in
raspimjpeg.conf
by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/272 - Add a GitHub Issue template for proposals by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/283
- Make the docs site easier to navigate by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/274
- build(deps-dev): bump pymdown-extensions from 10.3 to 10.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/285
- build(deps): bump caddy from 2.7.4 to 2.7.5 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/281
- build(deps-dev): bump mkdocs-material from 9.4.2 to 9.4.10 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/286
- build(deps-dev): bump poethepoet from 0.24.0 to 0.24.4 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/284
- build(deps-dev): remove mkdocs-material-extensions since it's deprecated by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/287
- build(deps-dev): bump mkdocs-macros-plugin from 1.0.4 to 1.0.5 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/280
- Add PDF export for schematic of custom HAT for v2.5 hardware by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/288
- Add a placeholder title for the proposal issue template by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/293
- Update operation docs for new URLs, landing page, and Node-RED changes by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/294
- Build a minimal version of the docs site, without hardware setup guides by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/295
- Fix the values of the
process_source
andprocess_commit
metadata fields by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/273 - Add v2.5 hardware BOM to v2.5 kit production guide by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/296
- Automatically deploy beta versions of the docs site to docs-beta.planktoscope.community by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/298
- Automatically deploy stable versions of the docs site to docs.planktoscope.community by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/299
- Fix the names of deployment env secrets for SSH deploy keys by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/300
- Update offline docs & landing page in the distro by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/301
- Bump version number of the docs package to 2023.9.0b3 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/308
- Bump version numbers for v2023.9.0-beta.2 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/309
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0-beta.1...software/v2023.9.0-beta.2
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli over 1 year ago
PlanktoScope - Software: Sunray v2023.9.0-beta.1
v2023.9.0 will be our first software update in over a year and a half, and we’re excited to make it easier for you to use your PlanktoScope in this update and in future updates! The v2023.9.0 software release focuses on improving the software infrastructure of the PlanktoScope software distro in preparation for future software improvements and maintenance, and making various usability improvements.
This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users.
Planning, management, and maintenance of the v2023.9.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina (@tpollina). If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
About this pre-release
v2023.9.0-beta.1 is a pre-release testing version for you to test out. It definitely has software bugs and other problems (the problems we know about are listed under the “Known Issues” section near the end of these release notes), so you should not use it for collecting scientific data! We would greatly appreciate it if you could flash the v2023.9.0-beta.1 image to an SD card and let us know about any bugs or problems you encounter while trying to use your PlanktoScope. This will help us to deliver a higher-quality software release with our v2023.9.0 release.
If you decide to try out v2023.9.0-beta.1, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Because this pre-release includes many changes, the changelogs below only list items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Changelog since v2.3
Highlights:
- To use the PlanktoScope from your web browser, now you should just open http://home.pkscope/ or http://pkscope.local/ or http://192.168.4.1/.
- You no longer need to use FileZilla to download and delete the datasets on your PlanktoScope - now you can do it from your web browser.
- The PlanktoScope now acts like a router, so if it has internet access then it will share that internet access with your computer/phone.
- PlanktoScopes will no longer stay connected to any Wi-Fi network which doesn’t provide internet access.
- PlanktoScopes now use a new naming scheme for machine names.
- We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
- The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
- Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.
Added:
- The PlanktoScope can now be accessed from your web browser at http://home.pkscope/ (assuming your web browser isn’t configured to use a custom DNS server such as
1.1.1.1
or8.8.8.8
) and http://pkscope.local/ (assuming your web browser supports mDNS), in addition to the other URLs which previously worked (http://planktoscope.local/, http://192.168.4.1/, and http://192.168.5.1/).- Each PlanktoScope can now also be accessed at machine-specific URLs of format
http://<machine-name>.pkscope
andhttp://pkscope-<machine-name>.local
, where<machine-name>
is replaced with your PlanktoScope’s machine name (see the note in the “Changed” section below about machine names). - If your web browser tries to do a Google search for the URL instead of actually opening it as a webpage, make sure you type
http://
at the start or/
at the end (e.g. home.pkscope/ or http://home.pkscope/ instead ofhome.pkscope
) - If you connect directly from your computer/phone to the PlanktoScope and then use your web browser to open http://home.pkscope/,
http://<machine-name>.pkscope
,http://pkscope-<machine-name>.local
, http://pkscope.local/, http://192.168.4.1/, or http://192.168.5.1/, you will now see a landing page with a list of useful links to software running on the PlanktoScope.
- Each PlanktoScope can now also be accessed at machine-specific URLs of format
- The PlanktoScope now shares any internet access it has from Wi-Fi with all devices connected to its Ethernet port; and it also shares any internet access it has from Ethernet with all devices connected to its Wi-Fi hotspot, if it’s operating in wireless AP mode.
- An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page. The online version of the documentation is now at https://planktoscope.github.io/PlanktoScope/
- The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2 . You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.
- The PlanktoScope documentation pages have been redone in a new system, and improved in various ways. The hardware documentation now describes the v2.5 PlanktoScope hardware. Thanks to Sebastian Wendel (@sourceindex) for starting this work, and for contributing lots of documentation!
Changed:
- The Node-RED dashboard (previously available at http://planktoscope.local:1880/ui) has been moved to a different URL, accessible by a link on the new landing page (see the note in the “Added” section about new URLs and the landing page).
- The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, edit text files, download folders as ZIP files, rename folders, and perform other common tasks.
- This way, you no longer have to install FileZilla in order to copy or delete any of your datasets!
- Previously, PlanktoScope machine names were generated as gibberish words like "Babaxio-Detuiau", and the machine names were used as the names of the Wi-Fi hotspot networks made by the PlanktoScope. However, the machine names created by this naming scheme were often difficult to pronounce, remember, and type for people in various languages, and the naming scheme sometimes generated names which sounded like curses or insults in some languages. Now, PlanktoScope machine names are generated as a combination of two words and a number up to five digits long; words are selected from pre-built lists in a language which can be chosen based on localization settings. Currently, word lists are only provided in US English, resulting in names like "metal-slope-23501", "conscious-pocket-1684", and "plant-range-10581"; however, word lists can be added for other languages in the future, and a user interface will eventually be provided for changing localization settings.
- This change makes it easier for you to tell someone else which PlanktoScope to connect to if multiple PlanktoScope Wi-Fi hotspot networks are available nearby.
- If you know a language besides US English and you’d like to help us add support for generating machine names in that language, please start a discussion on Github or in the
#6-dev-software
channel on the PlanktoScope Slack workspace!
- Previously, the PlanktoScope would stay connected to a wifi network even if it didn’t have internet access on that network. Now, if it connects to a Wi-Fi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to creating its own Wi-Fi hotspot for your computer or phone to connect to.
Deprecated:
- The
planktoscope.local
mDNS name is no longer recommended for use; instead, you should usepkscope.local
or the machine-specific mDNS name of formatpkscope-{machine-name}.local
.- However, we will continue to support
planktoscope.local
for a while, to maintain compatibility with earlier software versions.
- However, we will continue to support
- In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features.
- Depending on the amount of work required, we might be able to make the Node-RED dashboard for the custom PlanktoScope HAT be compatible with the Adafruit HAT. However, we do not have any specific plans yet.
- Regardless, we do plan to continue to fix bugs in the Node-RED dashboard for the Adafruit HAT for some time, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.
Fixed:
- When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file). Previously, invalid white balance values were written to the hardware.json configuration file, which would cause the PlanktoScope software to crash after every subsequent restart until the hardware.json file was manually removed or fixed.
- Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
- In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to show the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
- Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!
- The
adafruit-blinka
andadafruit-platformdetect
software dependencies are now updated to their latest version. As a result, the PlanktoScope software should not crash just because it is being used with recent versions of the Adafruit Stepper HAT.- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
pip3 install --upgrade adafruit-blinka adafruit-platformdetect
command before your PlanktoScope can work properly.
- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
Changelog since v2023.9.0-beta.0
Highlights:
- In the domain names and SSIDs for accessing PlanktoScopes, we’ve shortened the word “planktoscope” to “pkscope” (so e.g. use http://home.pkscope/ instead of http://home.planktoscope/, and use
http://pkscope-{machine-name}.local
instead ofhttp://planktoscope-{machine-name}.local
). - We fixed most of the regressions reported about v2023.9.0-beta.0 which we’ve been able to reproduce so far. We’ll need more people to help with testing in order for us to figure out the other problems which have been reported and which we haven’t yet been able to reproduce.
Added:
- The PlanktoScope can now be accessed with the mDNS name
pkscope.local
(so e.g. you can access the landing page at http://pkscope.local/ if you previously were able to use http://planktoscope.local/ ) - The Node-RED dashboard now provides some additional guidance about how to navigate the software logs, since it’s more complicated now.
- Thanks to Ana Fernandez Carrera on the PlanktoScope Slack for providing feedback which led to this change!
- The Node-RED dashboard now provides some additional warnings about what will happen if you touch the “Hardware Version” dropdown menu on the “Hardware Settings” page, since now the white balance settings get overwritten.
- Thanks to Fabien Lombard (@fabienlombard) for providing feedback which led to this change!
Changed:
- The top-level domain for domain names of format
home.planktoscope
and{machine-name}.planktoscope
(e.g. http://metal-slope-23501.planktoscope) has been changed to replaceplanktoscope
withpkscope
, so that the domain names are now of formathome.pkscope
and{machine-name}.pkscope
. Similarly, the machine-specific mDNS name has been changed from formatplanktoscope-{machine-name}.local
topkscope-{machine-name}.local
. - The SSIDs of wifi hotspots generated by the PlanktoScope has been changed from the format
PlanktoScope {machine-name}
to the formatpkscope-{machine-name}
. This makes it easier to determine the machine-specific mDNS URL: just add.local
, to getpkscope-{machine-name}.local
(e.g.pkscope-metal-slope-23501.local
).
Deprecated:
- The
planktoscope.local
mDNS name is no longer recommended for use; instead, you should usepkscope.local
or the machine-specific mDNS name of formatpkscope-{machine-name}.local
. However, we will continue to supportplanktoscope.local
for a while, to maintain compatibility with earlier software versions.
Fixed:
- The segmenter should now be able to start running, instead of crashing and becoming unresponsive.
- Thanks to Fabien Lombard (@fabienlombard) and Ana Fernandez Carrera on the PlanktoScope Slack for reporting this bug and helping us to troubleshoot it!
- The Node-RED dashboard should once again be able to receive information about the camera type from the Python backend when the Python backend starts.
- Thanks to Fabien Lombard (@fabienlombard) and Ana Fernandez Carrera on the PlanktoScope Slack for reporting this bug and helping us to troubleshoot it!
- The Node-RED dashboard should once again have a default pixel size calibration, so that it doesn’t produce raw image datasets which crash the segmenter.
- Thanks to Fabien Lombard (@fabienlombard) and Ana Fernandez Carrera on the PlanktoScope Slack for reporting this bug and helping us to troubleshoot it!
- Sometimes the PlanktoScope would generate a machine name which was so long that it prevented the Wi-Fi hotspot from being started; now, there’s a limit on how long the generated machine names can be.
- Thanks to Fabien Lombard (@fabienlombard) for reporting this bug and helping us to troubleshoot it!
- Machine-specific links on the landing page were broken; now they should be fixed.
- The offline documentation site was actually pinned at v2023.9.0-alpha.0; now it’s at v2023.9.0-beta.1, which (notably) includes fixes to some broken links to hardware design files.
Known Problems:
- (Known as of v2023.9.0-beta.0) In certain cases, some computers connected to both a Wi-Fi network providing internet access and to a PlanktoScope may be unable to access anything on the internet. If you experience this problem, please report it and your testing circumstances (e.g. your OS, your network configuration for internet access, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2.3, but potentially worse in v2023.9.0-beta.0) In certain cases, image acquisition will fail to start, with a “camera timeout” error. If you experience this problem, please report it and your testing circumstances (e.g. when would the error occur, how repeatable was it, etc.), to help us determine why this problem is happening for you, so that we can fix it!
- (Known as of v2023.9.0-beta.0) The Node-RED dashboard is too eager and aggressive in undoing any changes you make to the white balance which could cause an invalid white balance value. If you experience this problem, please report what you were doing (including the exact sequence of keys you pressed on the keyboard) and how the software behaved as a result, so that we can design a solution which works reasonably well for everyone!
- (Known as of v2023.9.0-beta.0) The landing page links to the old PlanktoScope documentation site at https://planktoscope.readthedocs.io/en/latest/ as the online documentation site, instead of linking to the new PlanktoScope documentation site. This is because we don’t yet have a URL for a stable version of the new documentation. For now, you should just use the landing page’s offline documentation link instead.
- (Known as of v2023.9.0-beta.0) The changelog in the offline documentation on the PlanktoScope is out-of-date. This will be fixed before the stable release of v2023.9.0.
- (Known as of v2023.9.0-beta.0) The PlanktoScope takes a while to start up and to reboot. This is a complicated problem, and we won’t be able to solve it before the v2023.9.0 stable release.
- (Known as of v2023.9.0-beta.0) If you restart the Node-RED dashboard (e.g. via Cockpit), you will also need to manually restart the hardware controller, or else the Node-RED dashboard won’t know the type of camera installed in the PlanktoScope. The solution will take some work and will not be delivered in the v2023.9.0 stable release.
- (Known as of v2.3) After restarting the hardware controller from the Node-RED dashboard’s “Administration” page, the Node-RED dashboard will report that the sample illumination LED is still on even though the hardware controller will have turned the LED off upon starting. The solution will take lots of work and will not be delivered in the v2023.9.0 stable release.
- (Known as of v2.3) The filenames of raw images produced by image acquisition can’t be sorted chronologically, because the filename includes acquisition time but not date. We plan to fix this in a future release after v2023.9.0.
Pull Requests
- Make v2.5 hardware files available by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/215
- Limit SSID length for wifi hotspot networks by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/214
- build(deps-dev): bump mkdocs-charts-plugin from 0.0.9 to 0.0.10 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/194
- build(deps-dev): bump pillow from 9.5.0 to 10.0.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/192
- Bump device-backend to use new machine naming scheme by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/216
- Fix some Node-RED usability papercuts with a limited scope of changes by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/223
- Bump the docs version number to v2023.9.0-beta.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/226
- Update setup scripts and bump version numbers for v2023.9.0-beta.1 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/227
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0-beta.0...software/v2023.9.0-beta.1
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli over 1 year ago
PlanktoScope - Software: Sunray v2023.9.0-beta.0
v2023.9.0 will be our first software update in over a year and a half, and we’re excited to make it easier for you to use your PlanktoScope in this update and in future updates! The v2023.9.0 software release focuses on improving the software infrastructure of the PlanktoScope software distro in preparation for future software improvements and maintenance, and making various usability improvements.
This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users.
Planning, management, and maintenance of the v2023.9.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina (@tpollina). If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag them in a new Github Discussions post or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
About this pre-release
v2023.9.0-beta.0 is a pre-release testing version for you to test out; it probably has software bugs and other problems, so you should not use it for collecting scientific data - expect bugs and software breakage! We would greatly appreciate it if you could flash the v2023.9.0-beta.0 image to an SD card and let us know about any bugs or problems you encounter while trying to use your PlanktoScope. This will help us to deliver a higher-quality software release with our v2023.9.0 release.
If you decide to try out v2023.9.0-beta.0, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Because this pre-release includes many changes, the changelogs below only list items we think users are likely to notice or to need to know about; for a full list of changes, please refer to software/CHANGELOG.md.
Changelog since v2.3
Highlights:
- To use the PlanktoScope from your web browser, now you should just open http://home.planktoscope/ or http://planktoscope.local/ or http://192.168.4.1/.
- You no longer need to use FileZilla to download and delete the datasets on your PlanktoScope - now you can do it from your web browser.
- The PlanktoScope now acts like a router, so if it has internet access then it will share that internet access with your computer/phone.
- PlanktoScopes will no longer stay connected to any Wi-Fi network which doesn’t provide internet access.
- PlanktoScopes now use a new naming scheme for machine names.
- We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
- The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
- Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.
Added:
- The PlanktoScope can now be accessed from your web browser at http://home.planktoscope/ (assuming your web browser isn’t configured to use a custom DNS server such as
1.1.1.1
or8.8.8.8
), in addition to the other URLs which previously worked (http://planktoscope.local/, http://192.168.4.1/, and http://192.168.5.1/).- Each PlanktoScope can now also be accessed at machine-specific URLs of format
http://<machine-name>.planktoscope
andhttp://planktoscope-<machine-name>.local
, where<machine-name>
is replaced with your PlanktoScope’s machine name (see the note in the “Changed” section below about machine names). - If your web browser tries to do a Google search for the URL instead of actually opening it as a webpage, make sure you type
http://
at the start or/
at the end (e.g. home.planktoscope/ or http://home.planktoscope/ instead ofhome.planktoscope
) - If you connect directly from your computer/phone to the PlanktoScope and then use your web browser to open http://home.planktoscope/,
http://<machine-name>.planktoscope
,http://planktoscope-<machine-name>.local
, http://planktoscope.local/, http://192.168.4.1/, or http://192.168.5.1/, you will now see a landing page with a list of useful links to software running on the PlanktoScope.
- Each PlanktoScope can now also be accessed at machine-specific URLs of format
- The PlanktoScope now shares any internet access it has from Wi-Fi with all devices connected to its Ethernet port; and it also shares any internet access it has from Ethernet with all devices connected to its Wi-Fi hotspot, if it’s operating in wireless AP mode.
- An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page. The online version of the documentation is now at https://planktoscope.github.io/PlanktoScope/
- The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2 . You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.
- The PlanktoScope documentation pages have been redone in a new system, and improved in various ways. The hardware documentation now describes the v2.5 PlanktoScope hardware. Thanks to Sebastian Wendel (@sourceindex) for starting this work, and for contributing lots of documentation!
Changed:
- The Node-RED dashboard (previously available at http://planktoscope.local:1880/ui) has been moved to a different URL, accessible by a link on the new landing page (see the note in the “Added” section about new URLs and the landing page).
- The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, edit text files, download folders as ZIP files, rename folders, and perform other common tasks.
- This way, you no longer have to install FileZilla in order to copy or delete any of your datasets!
- Previously, PlanktoScope machine names were generated as gibberish words like "Babaxio-Detuiau", and the machine names were used as the names of the Wi-Fi hotspot networks made by the PlanktoScope. However, the machine names created by this naming scheme were often difficult to pronounce, remember, and type for people in various languages, and the naming scheme sometimes generated names which sounded like curses or insults in some languages. Now, PlanktoScope machine names are generated as a combination of two words and a number up to five digits long; words are selected from pre-built lists in a language which can be chosen based on localization settings. Currently, word lists are only provided in US English, resulting in names like "metal-slope-23501", "conscious-pocket-1684", and "plant-range-10581"; however, word lists can be added for other languages in the future, and a user interface will eventually be provided for changing localization settings.
- This change makes it easier for you to tell someone else which PlanktoScope to connect to if multiple PlanktoScope Wi-Fi hotspot networks are available nearby.
- If you know a language besides US English and you’d like to help us add support for generating machine names in that language, please start a discussion on Github or in the
#6-dev-software
channel on the PlanktoScope Slack workspace!
- Previously, the PlanktoScope would stay connected to a wifi network even if it didn’t have internet access on that network. Now, if it connects to a Wi-Fi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to creating its own Wi-Fi hotspot for your computer or phone to connect to.
Deprecated:
- In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features. However, we will continue to fix bugs in the Node-RED dashboard for the Adafruit HAT, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.
Fixed:
- When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file). Previously, invalid white balance values were written to the hardware.json configuration file, which would cause the PlanktoScope software to crash after every subsequent restart until the hardware.json file was manually removed or fixed.
- Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
- In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to show the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
- Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!
- The
adafruit-blinka
andadafruit-platformdetect
software dependencies are now updated to their latest version. As a result, the PlanktoScope software should not crash just because it is being used with recent versions of the Adafruit Stepper HAT.- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
pip3 install --upgrade adafruit-blinka adafruit-platformdetect
command before your PlanktoScope can work properly.
- This fix removes the need for you to give your PlanktoScope internet access and then connect to the PlanktoScope over SSH to manually run the
Changelog since v2023.9.0-alpha.0
Highlights:
- You can now access offline copies of the quantitative imaging protocols for the PlanktoScope.
- We will eventually transition the Node-RED dashboard for the v2.1 hardware (Adafruit HAT) into maintenance mode.
- The Node-RED dashboard will no longer try to use incorrect URLs to show the camera preview stream.
- Entering invalid white balance values in the Node-RED dashboard should no longer break the PlanktoScope software upon restart.
Added:
- The PlanktoScope documentation now includes the reference protocol for quantitative imaging with PlanktoScope, which is normally available at https://www.protocols.io/view/planktoscope-protocol-for-plankton-imaging-bp2l6bq3zgqe/v2. You can access the protocol from the bottom of the “Usage” → “Getting Started” page of the PlanktoScope documentation.
Deprecated:
- In a future release (timeline not yet decided), the version of the Node-RED dashboard for the Adafruit HAT (hardware v2.1) will stop receiving new features even as the version of the Node-RED dashboard for the custom PlanktoScope HAT (hardware v2.3+) continues receiving new features. However, we will continue to fix bugs in the Node-RED dashboard for the Adafruit HAT, and we will continue to build SD card images for the Adafruit HAT which will also include new features in other software components.
Fixed:
- When an invalid value is entered for the red or blue white balance gain on the Node-RED dashboard's "Optic Configuration" page, that value is now ignored, a notification is displayed about the invalid value, and the white balance gain is reset to the last valid value (loaded from the hardware.json configuration file).
- Thanks to Gilles Orazi (@glx314) for filing an issue about this problem!
- In various use cases (e.g. when accessing the PlanktoScope over a mesh VPN, or in some bug reports by users accessing the PlanktoScope over Ethernet), the Node-RED dashboard would be unable to the MJPEG streams of the camera preview and the object segmentation preview, resulting in a potentially misleading error message being shown about the Python scripts. Now, the Node-RED dashboard uses URLs for the MJPEG streams which we expect to be accessible wherever the Node-RED dashboard itself is accessible.
- Thanks to Rodrigo Gonçalves on the PlanktoScope Slack for reporting a bug related to this issue and helping us to troubleshoot it!
Known Issues:
- The Node-RED dashboard’s “System Monitoring” page has an “Information” panel whose “Instrument Type” and “Camera Name” fields are often empty.
Pull Requests
- Properly manage versions of the backend's direct & indirect dependencies by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/175
- Fix docs links, improve software setup & usage docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/176
- build(deps-dev): bump poethepoet from 0.21.1 to 0.22.0 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/178
- build(deps-dev): bump mkdocs-material from 9.2.0 to 9.2.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/179
- Simplify the Docker image tag used for pushed Git tags by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/180
- Upgrade forklift to v0.3.0 (replacing Docker Swarm Mode with Docker Compose) by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/181
- Reduce the size of the build artifacts for the documentation site by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/184
- Set favicon of docs to the PlanktoScope logo by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/185
- Lock all dependencies of the Node-RED dashboard by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/186
- Unify support for different HAT types into the same branch of the repo by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/187
- Update the "Hardware Settings" and "System Monitoring" dashboard pages by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/191
- Update setup scripts and bump version numbers for v2023.9.0-beta.0 by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/196
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/software/v2023.9.0-alpha.0...software/v2023.9.0-beta.0
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli over 1 year ago
PlanktoScope - Software: Sunray v2023.9.0-alpha.0
v2023.9.0 will be our first software update in a year and a half, and we’re excited to make it easier for you to use your PlanktoScope in this update and in future updates!
The v2023.9.0 software release focuses on improving the software infrastructure of the PlanktoScope software distro in preparation for future software improvements and maintenance. The goal of this release is to fix some bugs in the existing software, and to take a few early steps towards enabling smaller and more frequent software updates in the future, so that we can deliver bugfixes and improvements to you on a more timely schedule, and so that you can apply those changes in a safer, easier, and simpler way.
This release’s name is “Sunray” (named after the Sunray Venus clam). We will keep this name for future versions until the next release which introduces big changes for users. The v2023.12.0 release, which is planned for December 2023, will introduce big changes for users (including requiring users to re-flash their SD cards with a new SD card image) and thus will have a different release name.
v2023.9.0-alpha.0 is an unstable testing version for the PlanktoScope development team and any advanced users who are feeling particularly adventurous. It definitely has undiscovered software bugs, breakage, and other problems, and we will be using this prerelease in order to discover those bugs. We will eventually make a v2023.9.0-beta.0 prerelease where more of those bugs should be discovered and fixed, for more people to test out. If you decide to try out v2023.9.0-alpha.0, please file bug reports on GitHub Issues and/or let us know about any bugs you discover in the #6-dev-software
channel on the PlanktoScope Slack workspace!
Planning, management, and maintenance of the v2023.9.0 software release is led by Ethan Li (@ethanjli), with approval and oversight from Thibaut Pollina. If this software release causes new problems for how you are using PlanktoScope, Ethan will take responsibility for addressing any new issues caused by this release - just tag Ethan in a new discussion on Github or in the #6-dev-software
channel on the PlanktoScope Slack workspace!
The changes listed below are just the ones we think users are likely to notice or to need to know about since the previous stable release (v2.3.0); for a full list of changes between each stable release and unstable prerelease, please refer to the software/CHANGELOG.md
file in this repository.
Changelog
Added:
- The PlanktoScope can now be accessed from your web browser using
home.planktoscope
(assuming your web browser isn’t configured to use a custom DNS server such as1.1.1.1
or8.8.8.8
), in addition to the other URLs which previously worked (planktoscope.local
,192.168.4.1
, and192.168.5.1
). Each PlanktoScope can now also be accessed at unique domain names of format<machine-name>.planktoscope
andplanktoscope-<machine-name>.local
, where<machine-name>
is replaced with your PlanktoScope’s machine name (see the note in the “Changed” section below about machine names). If you connect directly from your computer/phone to the PlanktoScope and then use your web browser to open http://home.planktoscope/,http://<machine-name>.planktoscope/
,http://planktoscope-<machine-name>.local/
, http://planktoscope.local/, http://192.168.4.1/, or http://192.168.5.1/, you will now see a landing page with a list of useful links to software running on the PlanktoScope.- Note that
home.planktoscope
and<machine-name>.planktoscope
will probably work even ifplanktoscope.local
andplanktoscope-<machine-name>.local
don’t work (e.g. if you’re on an Android device) - so you might not need to type192.168.4.1
on your phone anymore! - If your web browser tries to do a Google search for the URL instead of actually opening it as a URL, type
http://
at the start or add a/
at the end (e.g.http://home.planktoscope
orhome.planktoscope/
orhttp://home.planktoscope/
instead ofhome.planktoscope
) - If you’re a new user, you no longer have to remember to type http://planktoscope.local:1880/ui in order to access the dashboard - instead, you should just open http://home.planktoscope/ (or http://planktoscope.local/ or http://192.168.4.1/ or etc.) in your web browser and follow the “Node-RED dashboard” link to open the dashboard!
- If you have multiple PlanktoScopes (lucky you!) and you need to access them by SSH or FileZilla without having to deal with security warnings about different SSH host fingerprints on
home.planktoscope
orplanktoscope.local
or192.168.4.1
or192.168.5.1
, then you should use either the<machine-name>.planktoscope
domain name or theplanktoscope-<machine-name>.local
domain name - for example,ssh pi@metal-slope-23501.planktoscope
, orssh pi@planktoscope-plant-range-10581.local
- so that each PlanktoScope is accessed at a domain name with a different host fingerprint.
- Note that
- A web-browser computer administration panel called Cockpit is now available. It's accessible by a link from the PlanktoScope's landing page.
- This is provided to enable users to manage system settings easily (e.g. changing the password from
copepode
to something more secure), and to help users with troubleshooting problems. - Cockpit provides a web browser-based SSH terminal for remotely running commands, at http://ethan-dev-1.planktoscope.d.prakashlab.fluitans.org/admin/cockpit/system/terminal.
- Cockpit also allows you to inspect the logs of any system service, such as the autohotspot service.
- This is provided to enable users to manage system settings easily (e.g. changing the password from
- A web-browser system file manager for the PlanktoScope’s entire SD card is now available. It's accessible by a link from the PlanktoScope's landing page.
- Among many other things, it allows you to create, edit, rename, and delete text files.
- This way, you may be able to fix a broken system without having to re-flash your SD card - at least for certain kinds of breakage.
- An offline version of the PlanktoScope documentation is now available. It's accessible by a link from the PlanktoScope's landing page.
- This way, you can check the PlanktoScope’s documentation even if you don’t have internet access!
- We are now using a system for running containerized apps called Docker for deploying various programs (e.g. the system file manager, the PlanktoScope documentation, and the landing page) as part of the PlanktoScope software distribution.
- Docker provides a way to download and run containerized apps, and to upgrade and downgrade their versions. Containerized apps run in isolation, which makes them easier to troubleshoot, to update, and to uninstall cleanly. The isolation mechanism also helps us to limit the potential impact of any software bugs or security vulnerabilities. Most of the new apps we’re adding in this release (with the notable exception of Cockpit) are running as containerized apps.
- We are using Docker as a tool to make it more safe, simple, and easy for you to upgrade your PlanktoScope’s software, and for you to download and run other apps on your PlanktoScope.
- Note for advanced users: in this release we are running Docker in Swarm Mode, with the only member of the cluster being the Raspberry Pi itself; use of Swarm Mode should be considered an experimental feature, and we may stop using Swarm Mode in a future release (e.g. if we determine that the Python backend needs to be run as a privileged container in order to work in a container; see https://github.com/moby/swarmkit/issues/1030 & https://github.com/moby/moby/issues/43419 for details).
- A web-browser Docker management panel called Portainer is now available. It's accessible by a link from the PlanktoScope's landing page.
- Portainer is only provided for advanced users to inspect, manage, and troubleshoot containerized programs easily.
- Our use of the Portainer dashboard should be considered an experimental feature, and in a future release we will probably switch to a different default dashboard for monitoring and troubleshooting Docker containerized applications.
- If we switch to a different default dashboard, we may still provide an easy way for users to install Portainer on their PlanktoScopes - it just won’t be provided as part of the default software distribution provided by our SD card images. If you want to use Portainer and you want to a way to easily install Portainer in the future even if we remove it from our standard software distribution, please start a discussion on Github or in the
#6-dev-software
channel on the PlanktoScope Slack workspace!
- An experimental command-line tool called Forklift can now be run by advanced users on the PlanktoScope (e.g. via SSH or via Cockpit’s Terminal), for (re)configuring the set of Docker containerized applications running on the PlanktoScope. We will continue to improve Forklift in future releases of the PlanktoScope software distribution.
- As we add more functionality to Forklift, and as we make the tool more stable, we plan for it to eventually become an open “app store” system, with a simple web browser interface allowing users to add third-party (Docker containerized) programs and/or to easily replicate their custom software configuration across multiple PlanktoScopes.
- If you’re interested in trying out Forklift, or if you think Forklift might be useful for something you’re trying to do with PlanktoScope if certain features will be added to it, please start a discussion on Github or in the #6-dev-software channel on the PlanktoScope Slack workspace!
- For now, users interested in trying out Forklift should have some experience with using Git from the command-line (just for basic operations like clone, pull, push, etc.) and with using Docker, since Forklift’s command-line workflow is designed to have some familiarity for Git and Docker users.
Changed:
- The dashboard (previously available at http://planktoscope.local:1880/ui) and the dashboard editor (previously available at http://planktoscope.local:1880/) have been moved to different URLs, accessible by links on the new landing page (see the note in the “Added” section about new URLs and the landing page). These changes are in preparation for other changes we will be making for the v2023.9.0 release.
- This change will make it easier for you to simultaneously run multiple different dashboards, and/or multiple different versions of the same dashboard, in the future. If/when we release a new version of the dashboard with a different layout or a different user experience in the future, we’ll still be able to keep the previous version of the dashboard running on every PlanktoScope for a year or two in order to make the transition more gradual for you.
- Although the new URLs are longer, you’ll still be able to access them easily by just going to http://planktoscope.local/ in your browser and clicking a link on the landing page loaded by that URL.
- The file gallery (previously available at http://planktoscope.local/) has now been renamed to the dataset file manager and moved to a different URL, accessible by a link on the new landing page. It now allows you to delete folders, preview images, edit text files, download folders as ZIP files, rename folders, and perform other common tasks.
- This way, you no longer have to install FileZilla in order to copy or delete any of your datasets!
- Previously, the PlanktoScope would stay connected to a wifi network even if it didn’t have internet access on that network. Now, if it connects to a wifi network but can’t get internet access on that network (specifically, if it’s unable to find google.com), then it will revert to its default wifi hotspot behavior by creating its own isolated wifi network for your computer or phone to connect to.
- This change allows you to recover the ability to connect to your PlanktoScope over wifi in order to fix the settings of your PlanktoScope if you tried to make it connect to the internet over a wifi network which isn’t allowing it to connect to the internet. This is useful if, for example, you’re trying to access the PlanktoScope over a mesh VPN such as ZeroTier or Tailscale.
- Previously, PlanktoScope machine names were generated as gibberish phrases like “Babaxio-Detuiau”, which were often difficult for people to pronounce and could be difficult for people to type. Now, PlanktoScope machine names are generated as a sequence of two words (selected from two lists of source words) and a number (up to 5 digits long), resulting in names like “metal-slope-23501”, “conscious-pocket-1684”, and “plant-range-10581”. This release only has word lists to generate machine names in US English, but we have future plans to add a configuration option in the PlanktoScope software to change the language used for generating machine names. If you know a language besides US English and you’d like to help us add support for generating machine names in that language, please start a discussion on Github or in the
#6-dev-software
channel on the PlanktoScope Slack workspace!- This change makes it easier for you to tell someone else which PlanktoScope to connect to if multiple PlanktoScope wifi hotspot networks are available nearby.
- If you prefer the previous naming scheme, please start a discussion on Github or send a message in the
#6-dev-software
channel on the PlanktoScope Slack workspace asking us to make a pair of word lists which you can select for generating names in the previous style.
- Previously, every PlanktoScope had a hostname of
planktoscope
(which you could see if you accessed the PlanktoScope over SSH). Now, every PlanktoScope has a different hostname in the formatplanktoscope-<machine-name>
, e.g.planktoscope-metal-slope-23501
orplanktoscope-plant-range-10581
.- This change makes it easier for you to tell which PlanktoScope you are connected to over SSH if you have SSH access to multiple PlanktoScopes.
- This change was also necessary for us to make the PlanktoScope accessible via
planktoscope-<machine-name>.local
over any network which the PlanktoScope is connected to. See the note in the "Added" section above for more details.
- The PlanktoScope documentation pages have been redone in a new system, and improved in various ways.
- Thanks to Sebastian Wendel (@sourceindex) for starting this work, and for contributing lots of documentation!
- The process of generating SD card images is now mostly automated, and installation of software dependencies is now more reproducible.
- This change will make it easier for us to deliver software updates and bug fixes to you in a more timely manner, and it will help improve the reproducibility of the SD card images we upload for you to use.
Deprecated:
- The Node-RED flow editor and dashboard should no longer be accessed on port 1880 (e.g. using http://planktoscope.local:1880/ and http://planktoscope.local:1880/ui); in the v2024.7.0 release they will no longer be accessible on that port. Starting with this release, you should access them by clicking the corresponding links in the landing page (which is at e.g. http://home.planktoscope/ or http://planktoscope.local/)
- Note: in this prerelease, the Node-RED dashboard accessed from the landing has not yet been updated to use the new URLs for the camera preview and object segmentation preview streams. The dashboard will display a message claiming that the Python backend is broken, but this may not necessarily be true. You should probably be able to access those preview streams directly via the microscope preview and segmented object preview links in the PlanktoScope’s landing page, in the “Network APIs” section near the bottom of the page. This problem will be fixed before the v2023.9.0-beta.0 prerelease, so that the Node-RED dashboard will once again display the image streams correctly. Or instead you can still just try to access the Node-RED dashboard on port 1880, now with URL path
/ps/node-red-v2/ui/
(e.g. http://home.planktoscope:1880/ps/node-red-v2/ui/), and the preview streams will probably display correctly.
- Note: in this prerelease, the Node-RED dashboard accessed from the landing has not yet been updated to use the new URLs for the camera preview and object segmentation preview streams. The dashboard will display a message claiming that the Python backend is broken, but this may not necessarily be true. You should probably be able to access those preview streams directly via the microscope preview and segmented object preview links in the PlanktoScope’s landing page, in the “Network APIs” section near the bottom of the page. This problem will be fixed before the v2023.9.0-beta.0 prerelease, so that the Node-RED dashboard will once again display the image streams correctly. Or instead you can still just try to access the Node-RED dashboard on port 1880, now with URL path
Removed:
- The
planktoscope.local
domain name now only works from devices connected directly to the PlanktoScope as a wifi hotspot or with an Ethernet cable - in other words, it no longer works in any other networks (e.g. LANs or mesh VPNs) which the PlanktoScope is on. On those other networks, you should useplanktoscope-<machine-name>.local
instead ofplanktoscope.local
.- This change is needed in order to support the
planktoscope-<machine-name.local
domain name on all networks which the PlanktoScope is on, so that your computer can access multiple PlanktoScopes connected to the same network (e.g. a LAN or a mesh VPN) using mDNS names (any domain name ending in.local
is an mDNS name). This is because all of those PlanktoScopes previously competed with each other for theplanktoscope.local
name, and there is no consistent way to determine which PlanktoScope would actually be accessed viaplanktoscope.local
; but if each PlanktoScope takes its own unique domain name ending in.local
, then each one can be accessed in a predictable way. - We are only able to keep
planktoscope.local
working for devices connected directly to the PlanktoScope because avahi-daemon and avahi-publish are implemented in such a way thatplanktoscope.local
can only be resolved reliably to a single IP address ifplanktoscope-<machine-name>
is set as the PlanktoScope’s hostname (see https://github.com/lathiat/avahi/issues/40 for technical details). - Unfortunately, it’s difficult to make DNS and mDNS work consistently and correctly in all settings and devices. If our changes in this release cause problems for you, please start a discussion in the
#6-dev-software
channel on Slack or on our Github Discussions forum and we’ll try to help. You might also enjoy this haiku about troubleshooting problems in computer networks: https://blog.cloudflare.com/content/images/2020/05/image1-8.png
- This change is needed in order to support the
Fixed:
- The
adafruit-blinka
andadafruit-platformdetect
software dependencies are now updated to their latest version. As a result, the PlanktoScope software should not crash just because it is being used with recent versions of the Adafruit Stepper HAT.- This fix removes the need for you to connect to the PlanktoScope over SSH to manually run the
pip3 install --upgrade adafruit-blinka adafruit-platformdetect
command before your PlanktoScope can work properly.
- This fix removes the need for you to connect to the PlanktoScope over SSH to manually run the
Pull Requests
- Update some broken links and improve the documentation by @tpollina in https://github.com/PlanktoScope/PlanktoScope/pull/122
- added support for waveshare rev2.1 by @bahtibyte in https://github.com/PlanktoScope/PlanktoScope/pull/119
- Master hardware 1.2 by @gromain in https://github.com/PlanktoScope/PlanktoScope/pull/125
- Fix incorrect link to documentation in
README.md
by @dnberlin in https://github.com/PlanktoScope/PlanktoScope/pull/130 - Make shell scripts to automate in-situ setup from a stock SD card by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/137
- Integrate documentation improvements from @sourceindex by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/148
- build(deps-dev): bump mkdocs-macros-plugin from 0.7.0 to 1.0.1 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/152
- Try to deploy docs from master branch directly to GitHub Pages by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/154
- Try building a Docker image with Caddy to serve the docs by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/155
- Fix incorrect container image name by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/156
- Don't auto-assign issues to old project boards by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/153
- build(deps-dev): bump mkdocs-material from 9.1.14 to 9.1.15 in /documentation by @dependabot in https://github.com/PlanktoScope/PlanktoScope/pull/158
- Set up the standard Pallet package deployment on the SD card by @ethanjli in https://github.com/PlanktoScope/PlanktoScope/pull/157
New Contributors
- @bahtibyte made their first contribution in https://github.com/PlanktoScope/PlanktoScope/pull/119
- @dnberlin made their first contribution in https://github.com/PlanktoScope/PlanktoScope/pull/130
Full Changelog: https://github.com/PlanktoScope/PlanktoScope/compare/v2.3...software/v2023.9.0-alpha.0
Biosphere - Marine Life and Fishery
- Python
Published by ethanjli almost 2 years ago
PlanktoScope - v2.3
Direct link to image download:
Adafruit version (if in doubt, this is the one you want)
Custom hat version
Summary of changes:
- Segmenter: working version with basic and solid settings. It's not possible to change those settings for every run, but this will come.
- Network: you can connect directly to Ethernet now, it should work on any machine and OS you use.
- All dependencies were updated to their latest version
- OS is fully updated as of 2021-12-03, base version is RaspberryOS buster (the move to bullseye is planned for the next release)
- Various UI changes happened that should make your life easier (please help us and give us feedback on what isn't clear or missing).
- Culture mode: Location is forced set to the South pole. If you need another way of this to work, please make your voice heard! Time and date are set to current machine date and time, but can be modified before progressing to the next stage.
- Test mode: To be able to capture test runs, without having to worry about datetime, location. Location is forced set to the South pole (again), datetime is forced set to local machine time.
- Stream and machine access: For those of you that were having problems with the streams, or straight up with the access to the machine through http://planktoscope.local:1880/ui we should have solved most of the issues with Bonjour's installation and stuff like that. It's now not needed anymore. The machine can be accessed either through the above URI, through http://192.168.1.4:1880/ui if you are connecting directly to the machine WiFi network, or through http://192.168.1.5:1880/ui if you are connecting directly to the Ethernet port.
- A flurry of tiny bugs with Node-red not behaving exactly as expected were also fixed (#80 #82 #91 #87 #96 #98 #97 #74)
Limitations and known issues:
- Segmenter: can't be stopped (yet). Be careful if you start runs on more than 500 images, those can take a long long time.
- Imager: captured images are brighter than what we see on the stream in the optic configuration and capture tabs. There is an issue internally with the camera that we haven't been able to solve yet. Please be aware of that and set your exposition to a (slightly) darker image than you would otherwise.
- On the new hat: RTC is not working properly yet. Date is stuck in 2019 when there is no network connection or no GPS signal.
Full Changelog:
https://github.com/PlanktonPlanet/PlanktoScope/compare/v2.2.1...v2.3
Biosphere - Marine Life and Fishery
- Python
Published by gromain over 3 years ago
PlanktoScope - v2.2.1
Small update to bring some small improvements and tiny changes.
Mainly, this was caused by #80, we had to update some python packages to make sure newest Raspberry's would be properly detected.
We took this opportunity to fix tiny bugs in the flows and introduce a new feature. It's now possible to switch development branch. Yeah, useful only for us dev.
Download link: https://drive.google.com/file/d/1y0k_NDXN0yT5caZrXhjLYXgQVxu-D5ql/view?usp=sharing
For most people, just clicking on the "Update" button in the "System Administration" tab of the interface will do the trick.
Biosphere - Marine Life and Fishery
- Python
Published by gromain almost 4 years ago
PlanktoScope - v2.2
This is the release 2.2 of the PlanktoScope code.
The full release notes can be read at https://planktonscope.readthedocs.io/en/latest/changelog/#v22-released-2021-01-15.
The new image can be downloaded at https://drive.google.com/file/d/1fht8r7P6_bVsfIIwo7wnGLQ_1uxWCnos/view?usp=sharing
Biosphere - Marine Life and Fishery
- Python
Published by gromain about 4 years ago
PlanktoScope - v2.1
Image is available here: https://drive.google.com/file/d/1zOmbmXqt5uELQC0FTha1ndjJyMvehGSk/view?usp=sharing
Here is the guide on how to flash it: https://planktonscope.readthedocs.io/en/latest/easy_install/
What changed in this release:
Breaking changes
- There is no GUI on this image, it was created based on the Raspberry OS Lite, so no GUI.
- The first boot may take MORE THAN 5 MINUTES (due to the partition resizing to fill your SD Card), this is not a bug, it's a feature! So please wait for the WiFi network
PlanktoScope
to appear before restarting the machine in frustation! - The wifi network is now named
PlanktoScope
, and the password iscopepode
. - The default user is
pi
and the default password iscopepode
for this user. - The images captured and the segmentation output is now in the folder
/home/pi/data
.
Improvements
Under the hood, whole new code is running! There is still a small bug that is triggered when running long imaging sequences, if this happens, restart the machine! We are working on finding where the problem may comes from. This bug is chased here.
Another change: automatic wifi setup! If you setup your Raspberry to connect to a wifi network (by editing /etc/wpa_supplicant/wpa_supplicant.conf
) it will try to connect to this network before starting its hotspot. Also, if you connect it to a wired network via its Ethernet port, it will share this connection to devices connected to the hotspot!
The documentation has been updated to include more information about the ribbon assembly and how to create a backup of your sd card.
Biosphere - Marine Life and Fishery
- Python
Published by gromain over 4 years ago