John Chilton
083bdcbdd8
Vue-based component for defining collections by applying rules to a list of files or more general spreadsheet style information (e.g. sample sheets or tabular data from data sources containing URL or FTP file paths for files along with metadata). The widget is fairly complex but very broadly is broken into two panes - one to preview how rules are applied to build up tabular data defining collections (each row corresponding to a file with columns for metadata and such) and one that displays defined rules and allows for editing of these rules and creation of new ones. The goal behind defining rules this way instead of allowing the user to interact with the spreadsheet display directly is to enable scaling up collection creation. If a user wishes to upload hundreds of datasets - interacting with a widget directly for each input doesn't scale well and would be error prone. If a user wishes to upload hundreds of thousands of datasets - even loading this information in the GUI may not scale (though I've been impressed with the performance so far of this approach) and so we can potentially just display a preview of some of the rows and process the final set of rules on the backend. Since we can handle an arbitrary number of columns this way, we can define multiple list identifiers per file and so we can easily construct nested lists. Hence this allows creation of not just potentially larger collections but arbitrarily complex lists as well. Paired identifiers via indicator columns are also implemented. In order to operate over lists of datasets directly - the multi-select history widget now has a new option "Build Collection from Rules" along side the other collection builders. This mode uses the well established dataset collection API to build collections from HDAs. In order to operate on lists of FTP files or URLs - the upload widget has a new tab "Rule-based" tab that allows users to paste in tabular data or select a history dataset and then send this tabular data to the new builder widget. This will be extended to include FTP directories for instance over time. This mode uses the new data fetch API to build collections and handle uploads of arbitrary collections of files. The preview of the tabular data generated via rules is done via [Handsontable](https://handsontable.com/) - a JavaScript spreadsheet widget with a VueJS [wrapper component](https://github.com/handsontable/vue-handsontable-official). This turns out to be a fairly nice application for reactive components - as rules are added or modified the spreadsheet just naturally updates. In my hands the widget scales very nicely - I've uploaded files with tens of thousands of rows and rules modifying the data and changing the spreadsheet do not seem to cause siignificant delays in the web browser.
Client Build System
Installs, stages, and builds the client-side scripts necessary for running the Galaxy webapp. There's no need to use this system unless you are modifying or developing client-side scripts, or are running the development branch of Galaxy.
The base dependencies you will need are Node.js and Yarn. On OSX the easiest
way to get set up is using homebrew and the command brew install nodejs yarn.
More information including instructions for other platforms is available at
nodejs.org and yarnpkg.com.
The Galaxy client build has necessarily grown more complex in the past several years, but we're still trying to keep things as simple as possible for developers (everyone, really). If you're having any trouble with building the client after following the instructions below please create an issue on Github or reach out for help directly on Gitter at https://gitter.im/galaxyproject/Lobby.
Complete Client Build
There are many moving parts to the client build system, but the entrypoint for most people is the 'client' rule in the Makefile at the root of the Galaxy repository. Execute the following to perform a complete build suitable for local development, including dependency staging, style building, script processing and bundling.
make client
Note: Don't commit the files that 'make client' creates. This is a development-specific build which includes extra debugging features, and excludes several production optimizations made during the build process.
For a production build, suitable for committing to the repository, or deploying to a live server, use the following:
make client-production
And, lastly, if you want a production build that includes sourcemaps to allow for perusal of the page-level javascript to facilitate live-server debugging, use:
make client-production-maps
Automatic Rebuilding (Watch Mode)
When you're actively developing, it is sometimes convenient to have the client automatically rebuild every time you save a file. You can do this using:
make client-watch
This will first stage any dependencies (yarn-installed packages like jquery, etc), and then will watch for changes in any of the galaxy client source files. When a file is changed, the client will automatically rebuild, after which you can usually force refresh your browser to see changes. Note that it is still recommended to run 'make client' after you are finished actively developing using 'make client-watch'.
Changing Styles/CSS
The CSS and styling used by Galaxy is also controlled from this directory. Galaxy uses LESS, a superset of CSS that compiles to CSS, for its styling. LESS files are kept in client/galaxy/style/less. Compiled CSS is in static/style/blue.
As mentioned above, 'make client' will also rebuild styles. If you only want
to run the style task, use the following command from the client directory:
yarn run style