mirror of
https://github.com/galaxyproject/galaxy.git
synced 2026-09-24 16:30:27 +08:00
Merge branch 'ml-jupyter-notebook' of github.com:anuprulez/galaxy into ml-jupyter-notebook
This commit is contained in:
anup kumar
+1
-1
@@ -165,7 +165,7 @@ doc/build
|
||||
doc/schema.md
|
||||
doc/source/admin/config_logging_default_yaml.rst
|
||||
doc/source/dev/schema.md
|
||||
client/styleguide
|
||||
client/docs/dist
|
||||
|
||||
# webpack stats
|
||||
client/webpack-stats.json
|
||||
|
||||
@@ -164,138 +164,3 @@ terminal this starts for executing Jest tests.
|
||||
yarn run jest-watch Dialog
|
||||
|
||||
yarn run jest-watch workflow/run
|
||||
|
||||
### Writing a test file
|
||||
|
||||
Jest will try to test any file ending in "\*.test.js". Please place your test
|
||||
files inside client/src folders right next to whatever files that they are
|
||||
testing.
|
||||
|
||||
Jest has extensive documentation on the expect API, mocking, and more on the
|
||||
[official docs page](https://jestjs.io/docs/en/getting-started.html), which will
|
||||
be your best resource here.
|
||||
|
||||
```javascript
|
||||
// yourtestfile.test.js
|
||||
|
||||
import { things } from "./yourtestfile.js";
|
||||
|
||||
describe("some module you wrote", () => {
|
||||
let transientVariables;
|
||||
let serviceInstances;
|
||||
let testData;
|
||||
|
||||
beforeEach(() => {
|
||||
// setup your code (if necessary)
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
// teardown your code (so it doesn't ruin the next test)
|
||||
});
|
||||
|
||||
it("should do something or other", () => {
|
||||
expect(workflowNodeCount()).toBe(5);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Testing ~~Suggestions~~ Obligations
|
||||
|
||||
#### Clearly document the intent of your test
|
||||
|
||||
Please remember that these tests are not _for_ you. They're for the people who
|
||||
come after you. It will be a lot easier to modify, repair and upgrade your code
|
||||
if they can figure out what you were originally hoping to accomplish. Try to
|
||||
use as detailed 'expect' statements as possible -- overuse of 'toBeTruthy()'
|
||||
for example, can hide the intent of your test.
|
||||
|
||||
Add a couple of comments. Use variable names that mean something. Nobody's
|
||||
code is as self-documenting as they believe it to be.
|
||||
|
||||
|
||||
#### Only test the public API that you define (carefully!)
|
||||
|
||||
Internal implementations come and go with library upgrades and new tech. But
|
||||
the point of the unit test is to make sure your units work as designed....
|
||||
which means you need to... you know... design your code to work in units.
|
||||
|
||||
Separate your concerns and identify the developer-facing methods and functions
|
||||
you expect them to use. Test THOSE. Everything else should probably be
|
||||
considered an implementation detail.
|
||||
|
||||
The other side of the same coin is to test *only* the unit in question. If your
|
||||
component has a model that uses a service that touches Vuex, which then uses
|
||||
Axios to fetch some data -- don't test all that at once. Break things apart and
|
||||
mock functionality to isolate testing to units. End to end testing is a
|
||||
separate thing that shouldn't be attempted using spec tests in Jest.
|
||||
|
||||
Assume nobody cares _how_ your code works, we just need to know that the public
|
||||
API you designed _does_ work. If performance problems or new tech necessitate a
|
||||
re-write, these tests become a guide for the next implementation.
|
||||
|
||||
#### Wrap native browser resources in a function so they can be easily mocked
|
||||
|
||||
If your javascript needs to talk to the window object, or navigator, etc. wrap
|
||||
that in a function call so that it can be easily mocked during testing.
|
||||
|
||||
```javascript
|
||||
// myModule.js
|
||||
|
||||
// ... other code
|
||||
|
||||
export function redirectTo(url) {
|
||||
window.location = url;
|
||||
}
|
||||
|
||||
// myModule.test.js
|
||||
|
||||
jest.mock("myModule", () => ({
|
||||
redirectTo: (url) => {
|
||||
console.log(`I would have gone to: ${url}`);
|
||||
},
|
||||
}));
|
||||
|
||||
describe("some module", () => {
|
||||
// ....
|
||||
it("Performs a redirect when Foo is clicked"){
|
||||
expect(redirectTo.mock.calls.length).toBe(1);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
#### Implement logic in pure functions when possible
|
||||
|
||||
The more of your logic that is written in deterministic functions (i.e. no
|
||||
side-effects, same inputs always result in same outputs) the easier it is to
|
||||
test. Just load up the functions and supply suitable test inputs.
|
||||
|
||||
There is almost definitely no such thing as a well-written 1000 line function.
|
||||
Most of whatever happened in that thing was probably deterministic and can be
|
||||
broken up into easily testable chunks.
|
||||
|
||||
### Specific test scenarios & examples
|
||||
|
||||
[Mocking an imported
|
||||
dependency](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/tagService.test.js)
|
||||
|
||||
[Testing async
|
||||
operations](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/tagService.test.js)
|
||||
|
||||
[Testing a Vue component for expected rendering
|
||||
output](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/StatelessTags.test.js)
|
||||
|
||||
[Firing an event against a shallow mounted vue
|
||||
component](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/StatelessTags.test.js)
|
||||
|
||||
### The dirty secret about testing
|
||||
|
||||
It's good to have tests, but testing code isn't really about testing at all,
|
||||
it's actually about software design.
|
||||
|
||||
Maintainable software is testable software. If you can run a unit test on your
|
||||
code, then that means you must have necessarily separated your code into
|
||||
testable units and you will have almost definitely written better, more
|
||||
modular, more easily manipulated code, and nobody will ever contemplate using
|
||||
git-blame to figure out what went wrong.
|
||||
|
||||
Probably.
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
/**
|
||||
* Component root for rendering inside the styleguide, injects vuex store and other common elements
|
||||
* into each component example section.
|
||||
*/
|
||||
|
||||
import store from "../src/store";
|
||||
|
||||
export default (previewComponent) => {
|
||||
return {
|
||||
store,
|
||||
render(h) {
|
||||
return h(previewComponent);
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,142 @@
|
||||
/**
|
||||
* Functions for generating the sections for the style guide. Has the old style doc glob and a
|
||||
* recursive tree-walker that looks through the components folder and tries to arrange found
|
||||
* markdown docs in a tree.
|
||||
*/
|
||||
|
||||
const path = require("path");
|
||||
const fs = require("fs");
|
||||
const glob = require("glob");
|
||||
const { humanize, titleize } = require("underscore.string");
|
||||
|
||||
/**
|
||||
* Gets the table of contents sections for documentation in the components folder.
|
||||
*
|
||||
* @param {string} rootPath absolute path to components folder
|
||||
* @return {Object} Rootnode, section map, childMap
|
||||
*/
|
||||
function getDocSections(rootPath, options = {}) {
|
||||
const { ignore = [], docSelector = "*.@(vue|md)" } = options;
|
||||
|
||||
const sections = new Map(); // absolute section directory path -> section object
|
||||
const childToParent = new Map(); // absolute child path -> absolute parent path
|
||||
|
||||
// create root node
|
||||
const rootNode = newSection(rootPath, ignore);
|
||||
sections.set(rootPath, rootNode);
|
||||
childToParent.set(rootPath, null);
|
||||
|
||||
// get all matching doc files
|
||||
const selector = path.join(rootPath, "**", docSelector);
|
||||
const allFiles = glob.sync(selector, { ignore });
|
||||
|
||||
allFiles.forEach((file) => {
|
||||
const p = path.parse(file);
|
||||
|
||||
// intermediate dir names between this dir and root
|
||||
const relPath = path.relative(rootPath, p.dir);
|
||||
const midFolders = relPath.split(path.sep);
|
||||
|
||||
// build intermediate parent sections in the event that this is deeply-nested
|
||||
while (midFolders.length) {
|
||||
const sectionPath = path.join(rootPath, ...midFolders);
|
||||
if (!sections.has(sectionPath)) {
|
||||
// create new section
|
||||
const section = newSection(sectionPath, ignore);
|
||||
sections.set(sectionPath, section);
|
||||
// register relationship for tree-build later
|
||||
const parentPath = path.join(sectionPath, "..");
|
||||
childToParent.set(sectionPath, parentPath);
|
||||
}
|
||||
midFolders.pop();
|
||||
}
|
||||
|
||||
// if it's a MD file, add to parent as a subsection
|
||||
const section = sections.get(p.dir);
|
||||
if (section && isSubsection(section, file)) {
|
||||
section.children.add({
|
||||
name: titleize(humanize(p.name)),
|
||||
content: file,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// assemble recursive section tree under rootNode
|
||||
buildSectionTree(sections, childToParent);
|
||||
|
||||
// rootNode is the main result, returning the maps so the user has the option to manipulate the
|
||||
// tree before handing it over to the styleguide configs
|
||||
return { sections, childToParent, rootNode };
|
||||
}
|
||||
|
||||
function isSubsection(section, file) {
|
||||
const p = path.parse(file);
|
||||
|
||||
// it's not a subsection if it's not a markdown file
|
||||
if (p.ext !== ".md") return false;
|
||||
|
||||
// it's not a subsection if it's the same file as the section content
|
||||
if (file === section.content) return false;
|
||||
|
||||
// it's not a subsection if it's an example for an existing component
|
||||
const matchingComponentPath = path.join(p.dir, `${p.name}.vue`);
|
||||
const componentExists = fs.existsSync(matchingComponentPath);
|
||||
if (componentExists) return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new section in the TOC from passed path.
|
||||
*
|
||||
* @param {string} dir absolute directory path
|
||||
* @return {Object} section object
|
||||
*/
|
||||
function newSection(dir, ignore = []) {
|
||||
const section = {
|
||||
name: titleize(humanize(path.basename(dir))),
|
||||
components: () => glob.sync(path.join(dir, "*.vue"), { ignore }),
|
||||
children: new Set(),
|
||||
sectionDepth: 1,
|
||||
};
|
||||
|
||||
// summary doc is readme/docs/index.md
|
||||
const summarySelector = path.join(dir, "@(readme|docs|index).md");
|
||||
const summaryDocs = glob.sync(summarySelector, { ignore, nocase: true });
|
||||
if (summaryDocs.length) {
|
||||
section.content = summaryDocs[0];
|
||||
}
|
||||
|
||||
return section;
|
||||
}
|
||||
|
||||
/**
|
||||
* Turns the pair of maps into a nested object for use in the styleguide config script. Operates on
|
||||
* sections array by reference.
|
||||
*
|
||||
* @param {Map} sections Map of path -> section object
|
||||
* @param {Map} childToParent Map of childPath -> parentPath
|
||||
*/
|
||||
function buildSectionTree(sections, childToParent) {
|
||||
// add each child to its parent
|
||||
for (const [childPath, parentPath] of childToParent) {
|
||||
const child = sections.get(childPath);
|
||||
const parent = sections.get(parentPath);
|
||||
if (child && parent && parent.children) {
|
||||
parent.children.add(child);
|
||||
}
|
||||
}
|
||||
// convert child Sets to arrays now that de-dupe not required
|
||||
for (const section of sections.values()) {
|
||||
section.sections = Array.from(section.children).sort((a, b) => a.name.localeCompare(b.name));
|
||||
delete section.children;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
// public function
|
||||
getDocSections,
|
||||
// exporting these in case user wants to tweak the result tree and rebuild it
|
||||
buildSectionTree,
|
||||
newSection,
|
||||
};
|
||||
@@ -0,0 +1,73 @@
|
||||
Without a doubt. The most problematic part of the old client is the way every important piece of
|
||||
information was hung on a global variable whose initialization is completely unregulated.
|
||||
|
||||
### Don't directly reference window.Galaxy in Vue components
|
||||
|
||||
Components take props. Please pass in any values you might need from window.Galaxy as props and
|
||||
avoid referencing global Galaxy inside your components. I've even created basic providers which give
|
||||
you access to the Galaxy.config, current user, and current user histories. Please use them to
|
||||
retrieve your values, and bypass importing Galaxy altogether.
|
||||
|
||||
#### Sometimes you still need to update Vue from backbone as the legacy environment changes
|
||||
There are definitely use-cases where the Backbone models update over time and we need to update some
|
||||
value inside Vue. Instead of importing backbone models directly into Vue components, try building a
|
||||
backbone event listener that updates some relevant Vuex store.
|
||||
|
||||
* [Keeping Vuex in Sync with
|
||||
Galaxy](https://github.com/galaxyproject/galaxy/blob/dev/client/src/store/syncVuexToGalaxy.js)
|
||||
|
||||
These issues should disappear over time as the all of the old client is rebuilt in the new
|
||||
ecosystem.
|
||||
|
||||
|
||||
## Mount Functions
|
||||
|
||||
In what most people think of as a "standard" Vue application there would be only one place that Vue
|
||||
is mounted to the HTML environment, and that would be in a main.j or an app.js. Most modern
|
||||
single-page applications only have one starting point like that.
|
||||
|
||||
However, we are incrementally replacing old Backbone views, so in its current state, Galaxy may have
|
||||
several mounting functions for various components depending on where that component is intended to
|
||||
fit into the existing Backbone layouts.
|
||||
|
||||
|
||||
### Using the standard mount to pass in Galaxy variables as props
|
||||
|
||||
A standard mount function has been provided in src/utils. This mount function accepts a component
|
||||
definition, and then allows you to start up your Vue component with our standard load-out of plugins
|
||||
including localization, Vuex, and a couple other utilities. This is the preferred way to mount your
|
||||
component inside the old Backbone layout until the application is fully converted.
|
||||
|
||||
```js static
|
||||
// src/mvc/OldBackboneView.js
|
||||
|
||||
import { getGalaxyInstance } from "app";
|
||||
import MyComponent from "components/MyComponent";
|
||||
import { mountVueComponent } from "utils/mountVueComponent";
|
||||
|
||||
const OldBackboneView = {
|
||||
|
||||
someInitMethodYouMake() {
|
||||
const Galaxy = getGalaxyInstance();
|
||||
const mounter = mountVueComponent(MyComponent);
|
||||
|
||||
// pass in required props
|
||||
const props = {
|
||||
|
||||
// Something peeled off the global galaxy
|
||||
somePropVal: Galaxy.someDealie,
|
||||
|
||||
// ...or the current history
|
||||
name: Galaxy.currentHistory.name,
|
||||
|
||||
// or maybe from the backbone model for this view
|
||||
shoeSize: this.model.shoeSize
|
||||
};
|
||||
|
||||
// VM is a Vue instance.
|
||||
// this.$el is some jquery selection, first item is the actual DOM object
|
||||
const container = this.$el[0];
|
||||
const vm = mounter(props, container);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,18 @@
|
||||
Did you know, jQuery is old enough to drive? It's old enough to get a driver's license. jQuery is a
|
||||
tool that was built to deal with inconsistencies in browsers that NO LONGER EXIST. In a couple
|
||||
years, jQuery will be voting, drinking, and capable of being tried as an adult.
|
||||
|
||||
If you think you need jQuery, you are mistaken. Please seek help from somebody in the wg-ui-ux
|
||||
workgroup. There is nothing jQuery can provide you that isn't already part of vanilla javascript or
|
||||
a standard well-tested 3rd party modern npm module.
|
||||
|
||||
But that's not even the main problem. jQuery injects global initializations into every page on the
|
||||
site whether you want it or not. jQuery leverages an outdated initialization paradigm which is
|
||||
hugely problematic when it comes to unit testing and module building.
|
||||
|
||||
One of our most important goals in redesigning Galaxy is the complete elimination of this library
|
||||
from our source, along with all its invasive plugins.
|
||||
|
||||
### References
|
||||
* [You Don't Need jQuery](https://github.com/nefe/You-Dont-Need-jQuery)
|
||||
* [document.querySelector](https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector)
|
||||
@@ -0,0 +1,164 @@
|
||||
We are using components in two very distinct ways. The first, "normal", kind of component will
|
||||
probably look familiar to anybody whis is already passingly familiar with Vue. Here the relevant
|
||||
information comes in as properties, any internal variables get defined in "data", changes go out as
|
||||
events.
|
||||
|
||||
### Composition Example
|
||||
|
||||
```html static
|
||||
<!-- Use of a renderless component with a display component -->
|
||||
<DoodadProvider v-slot="{ doodad, saveDoodad }">
|
||||
<DoodadEditor :doodad="doodad" @update:doodad="saveDoodad" />
|
||||
</DoodadProvider>
|
||||
```
|
||||
|
||||
In this example, we've created a component whose job is to deal with loading and updating the doodad
|
||||
object. Notice that there is no markup inside the DoodadProvider other than the explicit renderless
|
||||
component we previously made, but you are free to putput whatever you want in there, accessing the
|
||||
doodad and saveDoddad properties as desired, as well as any other local data with the only
|
||||
restriction that Vue needs a single root element in which to render.
|
||||
|
||||
|
||||
## The Renderer
|
||||
|
||||
```html static
|
||||
<!-- DoodadEditor.vue, a simple "rendering" component -->
|
||||
|
||||
<template>
|
||||
<AutoComplete
|
||||
:options="options"
|
||||
:value="doodad.category"
|
||||
@select="saveCategory"
|
||||
/>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
|
||||
export default {
|
||||
props: {
|
||||
doodad: { type: Object, required: true },
|
||||
options: { type: Array, required: true },
|
||||
},
|
||||
methods: {
|
||||
saveCategory(newCategory) {
|
||||
this.$emit('update:doodad', { ...this.doodad, category: newCategory });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
</script>
|
||||
```
|
||||
|
||||
This component accepts a mandatory input object (doodad), lets the user play with a category prop,
|
||||
then emits a fresh object after it's done. So what, what's the big deal? The important part to walk
|
||||
away from this dumb example is the things that are NOT in this sample component.
|
||||
|
||||
This component doesn't save the data. This component doesn't make ajax calls, and this component
|
||||
doesn't mutate its props. What it does do is to allow the user to edit some object named "doodad"
|
||||
and emits a new fresh version of that doodad when it's done. (Note also that we are using the
|
||||
update:propname event syntax whenever possible [to facilitate .sync
|
||||
binds](https://vuejs.org/v2/guide/components-custom-events.html#sync-Modifier)).
|
||||
|
||||
Whatever happens to that new object is somebody else's job. As soon as you tie the data management
|
||||
to the rendering, the re-usability of your components craters.
|
||||
|
||||
|
||||
## The Provider
|
||||
|
||||
As the opposite of the rendering component, a provider or renderless component, is pure logic. It
|
||||
should not know or care what your renderer is going to do with the data it provides. It is simply a
|
||||
fancy way of configuring some data manipulation methods. This is one of the many ways of reusing
|
||||
functionality available in Vue. Some others are [Mixins](https://vuejs.org/v2/guide/mixins.html),
|
||||
[Provide/Inject](https://v3.vuejs.org/guide/component-provide-inject.html) and (in Vue3) [the
|
||||
composition API](https://v3.vuejs.org/guide/composition-api-introduction.html).
|
||||
|
||||
|
||||
```js static
|
||||
// DoodadProvider.js
|
||||
|
||||
import { loadDoodad, saveDoodad } from "./someAjaxQueryModule";
|
||||
|
||||
export default {
|
||||
data() {
|
||||
return {
|
||||
loaded: false,
|
||||
doodad: null
|
||||
}
|
||||
},
|
||||
methods: {
|
||||
async saveDoodad(newVal) {
|
||||
this.loaded = false;
|
||||
const newVal = await saveDoodad(newVal);
|
||||
this.doodad = newVal;
|
||||
this.loaded = true;
|
||||
},
|
||||
async loadDoodad(newVal) {
|
||||
this.loaded = false;
|
||||
const newVal = await loadDoodad(newVal);
|
||||
this.loaded = true;
|
||||
return newVal;
|
||||
}
|
||||
},
|
||||
async created() {
|
||||
this.doodad = await loadDoodad();
|
||||
},
|
||||
render() {
|
||||
return return this.$scopedSlots.default({
|
||||
loaded: this.loaded,
|
||||
doodad: this.doodad,
|
||||
saveDoodad: this.saveDoodad
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Here is an example of a simple possible renderless provider. The important part about this is the
|
||||
render() function which simply renders [one big default
|
||||
slot](https://vuejs.org/v2/guide/components-slots.html) and binds some of its own properties to that
|
||||
slot for use by downstream components.
|
||||
|
||||
## Unit Testing a renderless component
|
||||
|
||||
Testing of a component like our editor is pretty straightforward and follows the standard Vue
|
||||
guidelines. If you did a clean-enough job you won't even need to mock anything since all your data
|
||||
dependencies should be delivered via props.
|
||||
|
||||
But it's not so obvious how to unit-test a renderless provider. What do you check for? There's no
|
||||
mandatory markup, just one big empty slot.
|
||||
|
||||
```js static
|
||||
// Testing a renderless component
|
||||
|
||||
import { shallowMount } from "@vue/test-utils";
|
||||
import { getLocalVue, waitForLifecyleEvent } from "jest/helpers";
|
||||
import DoodadProvider from "./DoodadProvider";
|
||||
|
||||
describe("A renderless component", () => {
|
||||
const localVue = getLocalVue();
|
||||
let wrapper;
|
||||
let slotProps;
|
||||
|
||||
beforeEach(async () => {
|
||||
wrapper = shallowMount(DoodadProvider, {
|
||||
localVue,
|
||||
// The mount fn allows you to hook into a slot for this very reason
|
||||
scopedSlots: {
|
||||
default(props) {
|
||||
slotProps = props;
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
// waits for "updated" Vue lifecycle hook to fire on the renderless
|
||||
// component. This is often good enough for waiting for
|
||||
// an initial ajax load to finish, for example
|
||||
await waitForLifecyleEvent(wrapper.vm, "updated");
|
||||
})
|
||||
|
||||
test("someProp", () => {
|
||||
const { someProp } = slotProps;
|
||||
expect(someProp).toExist();
|
||||
// ...more tests
|
||||
})
|
||||
})
|
||||
```
|
||||
@@ -0,0 +1,71 @@
|
||||
### A component is really just a fancy function
|
||||
|
||||
I'm not talking about how webpack turns it into a rendering function. That's obvious.
|
||||
|
||||
I mean conceptually, props come in (like arguments) and events go out (like the return statements).
|
||||
A component is a fancy kind of function that can keep emitting results and accept changing inputs
|
||||
over time. In truth it more closely resembles an Observable, but an observable is ALSO a slightly
|
||||
fancier kind of function.
|
||||
|
||||
If you just think of a component as thing that takes input props and emits output events you're well
|
||||
on your way to using them well.
|
||||
|
||||
The worst components are ones that might as well just be a big single page script. That's zero
|
||||
percent better than the spaghetti we're working so hard to replace. That's just repackaging all the
|
||||
problems of the old imperative class-based legacy code.
|
||||
|
||||
### Get comfortable with events, limit your dependence on Vuex
|
||||
|
||||
New vue programmers are ok at handing props to components, but they rarely use events effectively
|
||||
(at first). As a result they end up using a lot of global state, a million little data props and
|
||||
relying on imperfect globalized tools like Vuex or other imported dependencies for every little
|
||||
variable.
|
||||
|
||||
Vuex definitely has its uses, but not as many as you might expect given the way it is
|
||||
overly-emphasized in common tutorials. It's easy to walk away from an "Intro to Vue" video with the
|
||||
idea that all data must live in Vuex all the time. That's a really undesirable situation.
|
||||
|
||||
Although vuex is a well-organized (many would say over-organized) state machine, it is important
|
||||
to remember that it is still a kind of global injection and deserves to be considered as such.
|
||||
|
||||
* [Should I Store This Data in
|
||||
Vuex](https://markus.oberlehner.net/blog/should-i-store-this-data-in-vuex/)
|
||||
* [Vuex getters are great, but don’t overuse
|
||||
them](https://codeburst.io/vuex-getters-are-great-but-dont-overuse-them-9c946689b414)
|
||||
|
||||
Data persistence should be something that happens near the top of your component tree, not down in
|
||||
the guts.
|
||||
|
||||
Your first thought with a component should be: "How can I offload the handling of the results of
|
||||
this component to my caller?" The answer is usually going to be events. A component that simply
|
||||
accepts props and emits events can be re-used in more contexts than one that relies on an external
|
||||
global state to operate.
|
||||
|
||||
#### Read up on .sync and v-model
|
||||
|
||||
They're just fancy shorthands for a prop / event handler combination. They are fundamentally no
|
||||
different from props and events, but the syntax is important to understand.
|
||||
|
||||
|
||||
### Think carefully about what should really be in "data".
|
||||
|
||||
Most of good component design boils down to answering the following question: What do I want to put
|
||||
in data, computed, and props?
|
||||
|
||||
Data is the place where temp data goes that is not sensible to persist in Vuex or other
|
||||
application-wide global state, usually because its use is very specific to the operation of this
|
||||
particular component. There really should only be a few variables in data.
|
||||
|
||||
#### Break down your own internal dependencies
|
||||
|
||||
Most components only need one or two variables in data. If you have a large amount of data
|
||||
variables, it's worth taking a little time to stop and build a mini-dependency tree for yourself.
|
||||
You will probably find that almost everything can be written in terms of computed transformations on
|
||||
a small number of data and properties.
|
||||
|
||||
Defining your values using computes will lead to a lot less "oh I forgot to update that
|
||||
variable"-style bugs.
|
||||
|
||||
If you have more than a few data variables you probably (a) aren't leveraging computeds and
|
||||
properties, or (b) are trying to implement too many features for just one component. It is important
|
||||
separate your concerns in components just like you do in any other kind of programming.
|
||||
@@ -0,0 +1,20 @@
|
||||
|
||||
[Galaxy uses Jest](https://jestjs.io/) for its client-side unit testing
|
||||
framework.
|
||||
|
||||
For testing Vue components, we use the [Vue testing
|
||||
utils](https://vue-test-utils.vuejs.org/) to mount individual components in a
|
||||
test bed and check them for rendered features. Please use jest-based mocking
|
||||
for isolating test functionality.
|
||||
|
||||
|
||||
### Specific test scenarios & examples
|
||||
|
||||
* [Mocking an imported
|
||||
dependency](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/tagService.test.js)
|
||||
* [Testing async
|
||||
operations](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/tagService.test.js)
|
||||
* [Testing a Vue component for expected rendering
|
||||
output](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/StatelessTags.test.js)
|
||||
* [Firing an event against a shallow mounted vue
|
||||
component](https://github.com/galaxyproject/galaxy/blob/dev/client/src/components/Tags/StatelessTags.test.js)
|
||||
@@ -0,0 +1,59 @@
|
||||
Part of making good code is making that code easy to test.
|
||||
|
||||
### Implement logic in pure functions when possible
|
||||
|
||||
The more of your logic that is written in deterministic functions (i.e. no
|
||||
side-effects, same inputs always result in same outputs) the easier it is to
|
||||
test. Just load up the functions and supply suitable test inputs.
|
||||
|
||||
There is almost definitely no such thing as a well-written 1000 line function.
|
||||
Most of whatever happened in that thing was probably deterministic and can be
|
||||
broken up into easily testable chunks.
|
||||
|
||||
|
||||
### Wrap native browser resources in a function so they can be easily mocked
|
||||
|
||||
If your javascript needs to talk to the window object, or navigator, etc. wrap
|
||||
that in a function call so that it can be easily mocked during testing.
|
||||
|
||||
|
||||
#### Your Module
|
||||
```js static
|
||||
// myModule.js
|
||||
|
||||
// ... other code
|
||||
|
||||
export function redirectTo(url) {
|
||||
window.location = url;
|
||||
}
|
||||
|
||||
export function theThingYouReallyCareAbout() {
|
||||
.....
|
||||
redirectTo(somePlace);
|
||||
....
|
||||
}
|
||||
```
|
||||
|
||||
#### Your test file
|
||||
```js static
|
||||
// myModule.test.js
|
||||
import { theThingYouReallyCareAbout, redirectTo } from "./myModule";
|
||||
|
||||
// calling jest.mock on the appropriate import path wraps the exports and gives
|
||||
// access to new testing methods like mockImplementation
|
||||
jest.mock("./myModule");
|
||||
redirectTo.mockImplementation((url) => {
|
||||
console.log(`I would have gone to: ${url}`);
|
||||
});
|
||||
|
||||
// You are testing theThingYouReallyCareAbout, but that function needs
|
||||
// to call a window.relocate, By wrapping redirectTo as a function it was easy to mock
|
||||
// and avoid errors in your test, because native browser objects do not even exist
|
||||
// in the unit-testing environment.
|
||||
describe("theThingYouReallyCareAbout", () => {
|
||||
it("Performs a redirect when Foo is clicked"){
|
||||
const result = theThingYouReallyCareAbout(abc123);
|
||||
expect(result).toBe(1);
|
||||
}
|
||||
});
|
||||
```
|
||||
@@ -0,0 +1,73 @@
|
||||
### Clearly document the intent of your test
|
||||
|
||||
Please remember that these tests are not _for_ you. They're for the people who
|
||||
come after you. It will be a lot easier to modify, repair and upgrade your code
|
||||
if they can figure out what you were originally hoping to accomplish. Try to
|
||||
use as detailed 'expect' statements as possible -- overuse of 'toBeTruthy()'
|
||||
for example, can hide the intent of your test.
|
||||
|
||||
Add a couple of comments. Use variable names that mean something. Nobody's
|
||||
code is as self-documenting as they believe it to be.
|
||||
|
||||
|
||||
### Only test the public API that you define
|
||||
|
||||
Internal implementations come and go with library upgrades and new tech. But
|
||||
the point of the unit test is to make sure your units work as designed....
|
||||
which means you need to... you know... design your code to work in units.
|
||||
|
||||
Separate your concerns and identify the developer-facing methods and functions
|
||||
you expect them to use. Test THOSE. Everything else should probably be
|
||||
considered an implementation detail.
|
||||
|
||||
The other side of the same coin is to test *only* the unit in question. If your
|
||||
component has a model that uses a service that touches Vuex, which then uses
|
||||
Axios to fetch some data -- don't test all that at once. Break things apart and
|
||||
mock functionality to isolate testing to units. End to end testing is a
|
||||
separate thing that shouldn't be attempted using spec tests in Jest.
|
||||
|
||||
Assume nobody cares _how_ your code works, we just need to know that the public
|
||||
API you designed _does_ work. If performance problems or new tech necessitate a
|
||||
re-write, these tests become a guide for the next implementation.
|
||||
|
||||
|
||||
### Writing a test file
|
||||
|
||||
Jest will try to test any file ending in "\*.test.js". Please place your test
|
||||
files inside client/src folders right next to whatever files that they are
|
||||
testing.
|
||||
|
||||
Jest has extensive documentation on the expect API, mocking, and more on the
|
||||
[official docs page](https://jestjs.io/docs/en/getting-started.html), which will
|
||||
be your best resource here.
|
||||
|
||||
```js static
|
||||
// yourcode.test.js
|
||||
|
||||
import { things } from "./yourcode.js";
|
||||
|
||||
describe("some module you wrote", () => {
|
||||
let transientVariables;
|
||||
let serviceInstances;
|
||||
let testData;
|
||||
|
||||
beforeEach(() => {
|
||||
// setup your code (if necessary)
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
// teardown your code (so it doesn't ruin the next test)
|
||||
});
|
||||
|
||||
it("should do something or other", () => {
|
||||
expect(workflowNodeCount()).toBe(5);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
### Check out the Jest helper functions
|
||||
|
||||
We have created some [common helpers for common testing
|
||||
scenarios](https://github.com/galaxyproject/galaxy/blob/dev/client/tests/jest/helpers.js).
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
# testdocs/a-file.md
|
||||
@@ -0,0 +1 @@
|
||||
# testdocs/another-file.md
|
||||
@@ -0,0 +1,3 @@
|
||||
<template>
|
||||
<div>Yay!</div>
|
||||
</template>
|
||||
@@ -0,0 +1 @@
|
||||
# ignored-file.md
|
||||
@@ -0,0 +1 @@
|
||||
# some-loose-file.md
|
||||
@@ -0,0 +1 @@
|
||||
# loose file
|
||||
@@ -0,0 +1 @@
|
||||
# testdocs/abc/readme.md
|
||||
@@ -0,0 +1 @@
|
||||
# foo
|
||||
@@ -0,0 +1 @@
|
||||
# another loose file
|
||||
@@ -0,0 +1 @@
|
||||
# Loose file
|
||||
@@ -0,0 +1,62 @@
|
||||
/**
|
||||
* Tests the functions which build the sections for the styleguide.
|
||||
*/
|
||||
|
||||
const path = require("path");
|
||||
const { getDocSections } = require("../sections");
|
||||
|
||||
const getSectionByName = (sections, name) => sections.find((o) => o.name == name);
|
||||
|
||||
describe("getDocSections", () => {
|
||||
const ignore = [
|
||||
// ignoring a whole directory
|
||||
"**/ignored/*",
|
||||
// ignoring a file
|
||||
"**/ignore*",
|
||||
];
|
||||
const testDocRoot = path.join(__dirname, "sample-docs");
|
||||
const { rootNode } = getDocSections(testDocRoot, { ignore });
|
||||
|
||||
test("section generation", () => {
|
||||
expect(rootNode.name).toEqual("Sample Docs");
|
||||
expect(rootNode.sections.length).toEqual(7);
|
||||
expect(rootNode.content).toBeUndefined();
|
||||
});
|
||||
|
||||
test("nested subsection should appear even if no docs in intermediate folders", () => {
|
||||
const deepSection = getSectionByName(rootNode.sections, "Nested Folders");
|
||||
expect(deepSection.sections.length).toEqual(1);
|
||||
});
|
||||
|
||||
test("subdirectory with readme should register as summary", () => {
|
||||
const subsection = getSectionByName(rootNode.sections, "Has Readme");
|
||||
|
||||
// readme interpreted as content file and not as loose section
|
||||
// one other loose file
|
||||
expect(subsection.content).toContain("readme.md");
|
||||
expect(subsection.sections.length).toEqual(1);
|
||||
});
|
||||
|
||||
test("subdirectory with no readme file", () => {
|
||||
const subsection = getSectionByName(rootNode.sections, "No Summary File");
|
||||
|
||||
// should just see 2 folders no summary
|
||||
expect(subsection.content).toBeUndefined();
|
||||
expect(subsection.sections.length).toEqual(2);
|
||||
});
|
||||
|
||||
test("subdirectory with ignored file", () => {
|
||||
const subsection = getSectionByName(rootNode.sections, "Contains Ommitted File");
|
||||
expect(subsection.sections.length).toEqual(1);
|
||||
});
|
||||
|
||||
test("ignored subdirectory", () => {
|
||||
const ignoredSection = getSectionByName(rootNode.sections, "Ignored");
|
||||
expect(ignoredSection).toBeUndefined();
|
||||
});
|
||||
|
||||
test("should not create a section for a component example file", () => {
|
||||
const section = getSectionByName(rootNode.sections, "Component Example");
|
||||
expect(section.sections.length).toEqual(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,69 @@
|
||||
### Beta History Panel Component Tree
|
||||
This is not intended to be a complete listing, but a general idea of how the components are intended
|
||||
to interact with each other.
|
||||
|
||||
```html static
|
||||
<CurrentHistoryPanel>
|
||||
<HistoryPanel :history="history">
|
||||
|
||||
<!-- for the right-hand side history we show some
|
||||
optional nav elements, can be ommitted for histories
|
||||
shown in multi-history view -->
|
||||
<slot:nav>
|
||||
<HistorySelector />
|
||||
<HistoryMenu />
|
||||
</slot:nav>
|
||||
|
||||
|
||||
<!-- if main history selected -->
|
||||
<History :history="history">
|
||||
|
||||
<!-- HCP does the heavy-lifting of mixing params, history, and
|
||||
scroll position to deliver the content for the scroller -->
|
||||
|
||||
<HistoryContentProvider :parent="history">
|
||||
<HistoryDetails />
|
||||
<HistoryMessages />
|
||||
<ContentOperations />
|
||||
<Scroller>
|
||||
|
||||
<!-- HistoryContentItem is a dynamic component that becomes
|
||||
either Dataset or DatasetCollection depending
|
||||
on the props passed to it -->
|
||||
|
||||
(<HistoryContentItem />)
|
||||
<Dataset />
|
||||
<!-- or -->
|
||||
<DatasetCollection />
|
||||
|
||||
</Scroller>
|
||||
</HistoryContentProvider>
|
||||
|
||||
</History>
|
||||
|
||||
|
||||
<!-- When a collection is selected for viewing, send in a
|
||||
breadcrumbs list of collections the user has selected -->
|
||||
|
||||
<CurrentCollection :selected-collections="breadcrumbs">
|
||||
<CollectionContentProvider :parent="selectedCollection">
|
||||
<CollectionNav />
|
||||
<Details />
|
||||
<Scroller>
|
||||
|
||||
<!-- Subdataset and Subcollection are similar to the Dataset
|
||||
and DatasetCollection ContentItem components, but mostly
|
||||
read-only since they are part of the collection-->
|
||||
|
||||
(<CollectionContentItem />)
|
||||
<Subdataset />
|
||||
<!-- or -->
|
||||
<Subcollection />
|
||||
|
||||
</Scroller>
|
||||
</CollectionContentProvider>
|
||||
</CurrentCollection>
|
||||
|
||||
</HistoryPanel>
|
||||
</CurrentHistoryPanel>
|
||||
```
|
||||
+1
-1
@@ -61,7 +61,7 @@
|
||||
<p class="text-center" v-if="is_unrestricted">
|
||||
You can
|
||||
<strong @click="toggleDatasetPrivacy(true)" class="make-private">
|
||||
<a href="javascript:void(0)">make this dataset private</a>
|
||||
<a id="make-private" href="javascript:void(0)">make this dataset private</a>
|
||||
</strong>
|
||||
to you.
|
||||
</p>
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@
|
||||
</h4>
|
||||
<b-row>
|
||||
<b-col>
|
||||
<div v-if="options && value">
|
||||
<div :class="permission_type" v-if="options && value">
|
||||
<multiselect
|
||||
v-model="value"
|
||||
:options="fetched_options"
|
||||
|
||||
@@ -109,6 +109,7 @@
|
||||
@onEdit="onEdit"
|
||||
@onAttributes="onAttributes"
|
||||
@onLint="onLint"
|
||||
@onUpgrade="onUpgrade"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
@@ -422,6 +423,9 @@ export default {
|
||||
this.$refs.lint.refresh();
|
||||
showLint();
|
||||
},
|
||||
onUpgrade() {
|
||||
this.attemptRefactor([{ action_type: "upgrade_all_steps" }]);
|
||||
},
|
||||
onEdit() {
|
||||
this.isCanvas = true;
|
||||
},
|
||||
|
||||
@@ -60,6 +60,9 @@
|
||||
<b-dropdown-item href="#" @click="$emit('onLayout')"
|
||||
><span class="fa fa-align-left mr-1" />Auto Layout</b-dropdown-item
|
||||
>
|
||||
<b-dropdown-item href="#" @click="$emit('onUpgrade')"
|
||||
><span class="fa fa-recycle mr-1" />Upgrade All Workflow Steps</b-dropdown-item
|
||||
>
|
||||
<b-dropdown-item href="#" @click="$emit('onDownload')"
|
||||
><span class="fa fa-download mr-1" />Download</b-dropdown-item
|
||||
>
|
||||
|
||||
+45
-46
@@ -1,58 +1,57 @@
|
||||
const path = require("path");
|
||||
const glob = require("glob");
|
||||
const fs = require("fs");
|
||||
const merge = require("webpack-merge");
|
||||
const baseConfig = require("./webpack.config.js");
|
||||
const { getDocSections } = require("./docs/sections");
|
||||
const buildWebpack = require("./webpack.config.js");
|
||||
|
||||
const webpackConfig = baseConfig();
|
||||
function getWebpack() {
|
||||
const cfg = buildWebpack();
|
||||
|
||||
const fileLoaderTest = /\.(png|jpg|jpeg|gif|svg|woff|woff2|ttf|eot)(\?.*$|$)/;
|
||||
// looks like our src plays with the webpack publicPath dynamically,
|
||||
// presumably to allow for dyamic loads, but this is a problem when
|
||||
// you're not outputting code to a non-standard location.
|
||||
// allowing this to happen breaks the styleguide.
|
||||
cfg.module.rules.push({
|
||||
test: /onload\/publicPath/,
|
||||
use: { loader: "ignore-loader" },
|
||||
});
|
||||
|
||||
const fileLoaderConfigRule = { rules: [{ test: fileLoaderTest, use: ["file-loader"] }] };
|
||||
return cfg;
|
||||
}
|
||||
|
||||
webpackConfig.module = merge.smart(webpackConfig.module, fileLoaderConfigRule);
|
||||
webpackConfig.output.publicPath = "";
|
||||
// TODO: Fix broken module imports before attempting to view in styleguidef
|
||||
const problemChildren = ["**/HistoryView.vue", "**/admin/DataManager/*", "**/LibraryFolder/*"];
|
||||
if (problemChildren.length) {
|
||||
console.warn("Not rendering styleguide for the following components:", problemChildren);
|
||||
}
|
||||
|
||||
webpackConfig.resolve.modules.push(path.join(__dirname, "src/style/scss"));
|
||||
function getSections() {
|
||||
// Style sections
|
||||
const docRootPath = path.join(__dirname, "docs/src");
|
||||
const { rootNode: docRoot } = getDocSections(docRootPath, { docSelector: "*.md" });
|
||||
const [design, styles] = docRoot.sections;
|
||||
delete docRoot.components;
|
||||
|
||||
const galaxyStyleDocs = [];
|
||||
glob.sync("./docs/galaxy-*.md").forEach((file) => {
|
||||
const name = file.match(/galaxy-(\w+).md/)[1];
|
||||
galaxyStyleDocs.push({ name: name, content: file });
|
||||
});
|
||||
// recursive component tree docs
|
||||
const cmpPath = path.join(__dirname, "src/components");
|
||||
const { rootNode: componentDocs } = getDocSections(cmpPath, { ignore: problemChildren });
|
||||
|
||||
const sections = [
|
||||
{
|
||||
name: "Galaxy styles",
|
||||
sections: galaxyStyleDocs,
|
||||
},
|
||||
{
|
||||
name: "Basic Bootstrap Styles",
|
||||
content: "./docs/bootstrap.md",
|
||||
},
|
||||
{
|
||||
name: "Components",
|
||||
// Components that are directories will get their own section
|
||||
sections: glob
|
||||
.sync("./src/components/*")
|
||||
.map((file) => {
|
||||
if (fs.lstatSync(file).isDirectory()) {
|
||||
return {
|
||||
name: path.basename(file),
|
||||
components: file + "/**/*.vue",
|
||||
};
|
||||
}
|
||||
})
|
||||
.filter((v) => v),
|
||||
// ...while top level components are handled here.
|
||||
components: "./src/components/*.vue",
|
||||
},
|
||||
];
|
||||
return [design, styles, componentDocs];
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
webpackConfig,
|
||||
webpackConfig: getWebpack(),
|
||||
title: "Galaxy Client Resources",
|
||||
sections: getSections(),
|
||||
getExampleFilename(componentPath) {
|
||||
return componentPath.replace(/\.(vue|js)?$/, ".md");
|
||||
},
|
||||
require: [
|
||||
"./src/style/scss/base.scss",
|
||||
"./src/polyfills.js",
|
||||
// "./src/bundleEntries.js"
|
||||
],
|
||||
tocMode: "collapse",
|
||||
renderRootJsx: "./docs/root",
|
||||
styleguideDir: "./docs/dist",
|
||||
pagePerSection: true,
|
||||
sections,
|
||||
require: ["./src/style/scss/base.scss", "./src/polyfills.js", "./src/bundleEntries.js"],
|
||||
vuex: "./src/store/index.js",
|
||||
ignore: problemChildren,
|
||||
};
|
||||
|
||||
@@ -118,7 +118,7 @@ module.exports = {
|
||||
rootDir: path.join(__dirname, "../../"),
|
||||
|
||||
// A list of paths to directories that Jest should use to search for files in
|
||||
roots: ["<rootDir>/src", "<rootDir>/tests/jest/standalone/"],
|
||||
roots: ["<rootDir>/src", "<rootDir>/tests/jest/standalone/", "<rootDir>/docs/"],
|
||||
|
||||
// Allows you to use a custom runner instead of Jest's default test runner
|
||||
// runner: "jest-runner",
|
||||
|
||||
@@ -53,7 +53,7 @@ function addIframe() {
|
||||
fetch("/training-material/")
|
||||
.then((response) => {
|
||||
if (!response.ok) {
|
||||
url = "https://training.galaxyproject.org/training-material/";
|
||||
url = "https://training.galaxyproject.org/training-material/?utm_source=webhook&utm_medium=noproxy&utm_campaign=gxy";
|
||||
message = `
|
||||
<span>
|
||||
<a href="https://docs.galaxyproject.org/en/master/admin/special_topics/gtn.html">Click to run</a> unavailable.
|
||||
@@ -70,7 +70,7 @@ function addIframe() {
|
||||
onloadscroll = storedLocation.split(" ")[0];
|
||||
url = storedLocation.split(" ")[1];
|
||||
} else {
|
||||
url = "/training-material/";
|
||||
url = "/training-material/?utm_source=webhook&utm_medium=proxy&utm_campaign=gxy";
|
||||
}
|
||||
message = "";
|
||||
}
|
||||
|
||||
+69
-47
@@ -4,13 +4,18 @@ import sys
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm.scoping import (
|
||||
scoped_session,
|
||||
)
|
||||
|
||||
import galaxy.model
|
||||
import galaxy.model.security
|
||||
import galaxy.queues
|
||||
import galaxy.security
|
||||
from galaxy import config, jobs
|
||||
from galaxy import auth, config, jobs
|
||||
from galaxy.config_watchers import ConfigWatchers
|
||||
from galaxy.containers import build_container_interfaces
|
||||
from galaxy.datatypes.registry import Registry
|
||||
from galaxy.files import ConfiguredFileSources
|
||||
from galaxy.job_metrics import JobMetrics
|
||||
from galaxy.managers.api_keys import ApiKeyManager
|
||||
@@ -19,25 +24,30 @@ from galaxy.managers.folders import FolderManager
|
||||
from galaxy.managers.hdas import HDAManager
|
||||
from galaxy.managers.histories import HistoryManager
|
||||
from galaxy.managers.interactivetool import InteractiveToolManager
|
||||
from galaxy.managers.jobs import JobSearch
|
||||
from galaxy.managers.libraries import LibraryManager
|
||||
from galaxy.managers.roles import RoleManager
|
||||
from galaxy.managers.session import GalaxySessionManager
|
||||
from galaxy.managers.tools import DynamicToolManager
|
||||
from galaxy.managers.users import UserManager
|
||||
from galaxy.managers.workflows import (
|
||||
WorkflowContentsManager,
|
||||
WorkflowsManager,
|
||||
)
|
||||
from galaxy.model.base import SharedModelMapping
|
||||
from galaxy.model.database_heartbeat import DatabaseHeartbeat
|
||||
from galaxy.model.mapping import GalaxyModelMapping
|
||||
from galaxy.model.tags import GalaxyTagHandler
|
||||
from galaxy.queue_worker import (
|
||||
GalaxyQueueWorker,
|
||||
send_local_control_task,
|
||||
)
|
||||
from galaxy.quota import get_quota_agent
|
||||
from galaxy.quota import get_quota_agent, QuotaAgent
|
||||
from galaxy.security.idencoding import IdEncodingHelper
|
||||
from galaxy.tool_shed.galaxy_install.installed_repository_manager import InstalledRepositoryManager
|
||||
from galaxy.tool_shed.galaxy_install.update_repository_manager import UpdateRepositoryManager
|
||||
from galaxy.tool_util.deps.views import DependencyResolversView
|
||||
from galaxy.tool_util.verify import test_data
|
||||
from galaxy.tool_util.verify.test_data import TestDataResolver
|
||||
from galaxy.tools.cache import (
|
||||
ToolCache,
|
||||
ToolShedRepositoryCache
|
||||
@@ -45,7 +55,7 @@ from galaxy.tools.cache import (
|
||||
from galaxy.tools.data_manager.manager import DataManagers
|
||||
from galaxy.tools.error_reports import ErrorReports
|
||||
from galaxy.tools.special_tools import load_lib_tools
|
||||
from galaxy.tours import build_tours_registry
|
||||
from galaxy.tours import build_tours_registry, ToursRegistry
|
||||
from galaxy.util import (
|
||||
ExecutionTimer,
|
||||
heartbeat,
|
||||
@@ -56,19 +66,23 @@ from galaxy.visualization.genomes import Genomes
|
||||
from galaxy.visualization.plugins.registry import VisualizationsRegistry
|
||||
from galaxy.web import url_for
|
||||
from galaxy.web.proxy import ProxyManager
|
||||
from galaxy.web_stack import application_stack_instance
|
||||
from galaxy.web_stack import application_stack_instance, ApplicationStack
|
||||
from galaxy.webhooks import WebhooksRegistry
|
||||
from galaxy.workflow.trs_proxy import TrsProxy
|
||||
from .structured_app import StructuredApp
|
||||
from .di import Container
|
||||
from .structured_app import BasicApp, StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
app = None
|
||||
|
||||
|
||||
class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin, Container):
|
||||
"""Encapsulates the state of a Universe application"""
|
||||
|
||||
def __init__(self, **kwargs) -> None:
|
||||
super().__init__()
|
||||
self._register_singleton(BasicApp, self)
|
||||
self._register_singleton(StructuredApp, self)
|
||||
if not log.handlers:
|
||||
# Paste didn't handle it, so we need a temporary basic log
|
||||
# configured. The handler added here gets dumped and replaced with
|
||||
@@ -80,19 +94,19 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
startup_timer = ExecutionTimer()
|
||||
self.new_installation = False
|
||||
# Read config file and check for errors
|
||||
self.config: Any = config.Configuration(**kwargs)
|
||||
self.config: Any = self._register_singleton(config.Configuration, config.Configuration(**kwargs))
|
||||
self.config.check()
|
||||
config.configure_logging(self.config)
|
||||
self.execution_timer_factory = ExecutionTimerFactory(self.config)
|
||||
self.execution_timer_factory = self._register_singleton(ExecutionTimerFactory, ExecutionTimerFactory(self.config))
|
||||
self.configure_fluent_log()
|
||||
# A lot of postfork initialization depends on the server name, ensure it is set immediately after forking before other postfork functions
|
||||
self.application_stack = application_stack_instance(app=self)
|
||||
self.application_stack = self._register_singleton(ApplicationStack, application_stack_instance(app=self))
|
||||
self.application_stack.register_postfork_function(self.application_stack.set_postfork_server_name, self)
|
||||
self.config.reload_sanitize_allowlist(explicit='sanitize_allowlist_file' in kwargs)
|
||||
self.amqp_internal_connection_obj = galaxy.queues.connection_from_config(self.config)
|
||||
# queue_worker *can* be initialized with a queue, but here we don't
|
||||
# want to and we'll allow postfork to bind and start it.
|
||||
self.queue_worker = GalaxyQueueWorker(self)
|
||||
self.queue_worker = self._register_singleton(GalaxyQueueWorker, GalaxyQueueWorker(self))
|
||||
|
||||
self._configure_tool_shed_registry()
|
||||
self._configure_object_store(fsmon=True)
|
||||
@@ -105,23 +119,30 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
|
||||
# Security helper
|
||||
self._configure_security()
|
||||
self._register_singleton(IdEncodingHelper, self.security)
|
||||
self._register_singleton(SharedModelMapping, self.model)
|
||||
self._register_singleton(GalaxyModelMapping, self.model)
|
||||
self._register_singleton(scoped_session, self.model.context)
|
||||
# Tag handler
|
||||
self.tag_handler = GalaxyTagHandler(self.model.context)
|
||||
self.dataset_collections_service = DatasetCollectionManager(self)
|
||||
self.history_manager = HistoryManager(self)
|
||||
self.hda_manager = HDAManager(self)
|
||||
self.workflow_manager = WorkflowsManager(self)
|
||||
self.workflow_contents_manager = WorkflowContentsManager(self)
|
||||
self.dependency_resolvers_view = DependencyResolversView(self)
|
||||
self.test_data_resolver = test_data.TestDataResolver(file_dirs=self.config.tool_test_data_directories)
|
||||
self.library_folder_manager = FolderManager()
|
||||
self.library_manager = LibraryManager()
|
||||
self.role_manager = RoleManager(self)
|
||||
self.dynamic_tool_manager = DynamicToolManager(self)
|
||||
self.api_keys_manager = ApiKeyManager(app)
|
||||
self.tag_handler = self._register_singleton(GalaxyTagHandler)
|
||||
self.user_manager = self._register_singleton(UserManager)
|
||||
self._register_singleton(GalaxySessionManager)
|
||||
self.hda_manager = self._register_singleton(HDAManager)
|
||||
self.history_manager = self._register_singleton(HistoryManager)
|
||||
self.job_search = self._register_singleton(JobSearch)
|
||||
self.dataset_collections_service = self._register_singleton(DatasetCollectionManager)
|
||||
self.workflow_manager = self._register_singleton(WorkflowsManager)
|
||||
self.workflow_contents_manager = self._register_singleton(WorkflowContentsManager)
|
||||
self.dependency_resolvers_view = self._register_singleton(DependencyResolversView, DependencyResolversView(self))
|
||||
self.test_data_resolver = self._register_singleton(TestDataResolver, TestDataResolver(file_dirs=self.config.tool_test_data_directories))
|
||||
self.library_folder_manager = self._register_singleton(FolderManager)
|
||||
self.library_manager = self._register_singleton(LibraryManager)
|
||||
self.role_manager = self._register_singleton(RoleManager)
|
||||
self.dynamic_tool_manager = self._register_singleton(DynamicToolManager)
|
||||
self.api_keys_manager = self._register_singleton(ApiKeyManager)
|
||||
|
||||
# ConfiguredFileSources
|
||||
self.file_sources = ConfiguredFileSources.from_app_config(self.config)
|
||||
self.file_sources = self._register_singleton(ConfiguredFileSources, ConfiguredFileSources.from_app_config(self.config))
|
||||
|
||||
# Tool Data Tables
|
||||
self._configure_tool_data_tables(from_shed_config=False)
|
||||
@@ -129,36 +150,37 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
self._configure_genome_builds(data_table_name="__dbkeys__", load_old_style=True)
|
||||
|
||||
# Genomes
|
||||
self.genomes = Genomes(self)
|
||||
self.genomes = self._register_singleton(Genomes)
|
||||
# Data providers registry.
|
||||
self.data_provider_registry = DataProviderRegistry()
|
||||
self.data_provider_registry = self._register_singleton(DataProviderRegistry)
|
||||
|
||||
# Initialize job metrics manager, needs to be in place before
|
||||
# config so per-destination modifications can be made.
|
||||
self.job_metrics = JobMetrics(self.config.job_metrics_config_file, app=self)
|
||||
self.job_metrics = self._register_singleton(JobMetrics, JobMetrics(self.config.job_metrics_config_file, app=self))
|
||||
|
||||
# Initialize error report plugins.
|
||||
self.error_reports = ErrorReports(self.config.error_report_file, app=self)
|
||||
self.error_reports = self._register_singleton(ErrorReports, ErrorReports(self.config.error_report_file, app=self))
|
||||
|
||||
# Initialize the job management configuration
|
||||
self.job_config = jobs.JobConfiguration(self)
|
||||
self.job_config = self._register_singleton(jobs.JobConfiguration)
|
||||
|
||||
# Setup a Tool Cache
|
||||
self.tool_cache = ToolCache()
|
||||
self.tool_shed_repository_cache = ToolShedRepositoryCache(self)
|
||||
self.tool_cache = self._register_singleton(ToolCache)
|
||||
self.tool_shed_repository_cache = self._register_singleton(ToolShedRepositoryCache)
|
||||
# Watch various config files for immediate reload
|
||||
self.watchers = ConfigWatchers(self)
|
||||
self.watchers = self._register_singleton(ConfigWatchers)
|
||||
self._configure_tool_config_files()
|
||||
self.installed_repository_manager = InstalledRepositoryManager(self)
|
||||
self.installed_repository_manager = self._register_singleton(InstalledRepositoryManager, InstalledRepositoryManager(self))
|
||||
self._configure_datatypes_registry(self.installed_repository_manager)
|
||||
self._register_singleton(Registry, self.datatypes_registry)
|
||||
galaxy.model.set_datatypes_registry(self.datatypes_registry)
|
||||
|
||||
self._configure_toolbox()
|
||||
|
||||
# Load Data Manager
|
||||
self.data_managers = DataManagers(self)
|
||||
self.data_managers = self._register_singleton(DataManagers)
|
||||
# Load the update repository manager.
|
||||
self.update_repository_manager = UpdateRepositoryManager(self)
|
||||
self.update_repository_manager = self._register_singleton(UpdateRepositoryManager, UpdateRepositoryManager(self))
|
||||
# Load proprietary datatype converters and display applications.
|
||||
self.installed_repository_manager.load_proprietary_converters_and_display_applications()
|
||||
# Load datatype display applications defined in local datatypes_conf.xml
|
||||
@@ -171,26 +193,26 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
load_lib_tools(self.toolbox)
|
||||
self.toolbox.persist_cache(register_postfork=True)
|
||||
# visualizations registry: associates resources with visualizations, controls how to render
|
||||
self.visualizations_registry = VisualizationsRegistry(
|
||||
self.visualizations_registry = self._register_singleton(VisualizationsRegistry, VisualizationsRegistry(
|
||||
self,
|
||||
directories_setting=self.config.visualization_plugins_directory,
|
||||
template_cache_dir=self.config.template_cache_path)
|
||||
template_cache_dir=self.config.template_cache_path))
|
||||
# Tours registry
|
||||
self.tour_registry = build_tours_registry(self.config.tour_config_dir)
|
||||
tour_registry = build_tours_registry(self.config.tour_config_dir)
|
||||
self.tour_registry = tour_registry
|
||||
self[ToursRegistry] = tour_registry # type: ignore
|
||||
# Webhooks registry
|
||||
self.webhooks_registry = WebhooksRegistry(self.config.webhooks_dir)
|
||||
self.webhooks_registry = self._register_singleton(WebhooksRegistry, WebhooksRegistry(self.config.webhooks_dir))
|
||||
# Load security policy.
|
||||
self.security_agent = self.model.security_agent
|
||||
self.host_security_agent = galaxy.model.security.HostAgent(
|
||||
model=self.security_agent.model,
|
||||
permitted_actions=self.security_agent.permitted_actions)
|
||||
# Load quota management.
|
||||
self.quota_agent = get_quota_agent(self.config, self.model)
|
||||
self.quota_agent = self._register_singleton(QuotaAgent, get_quota_agent(self.config, self.model))
|
||||
# Heartbeat for thread profiling
|
||||
self.heartbeat = None
|
||||
from galaxy import auth
|
||||
self.auth_manager = auth.AuthManager(self)
|
||||
self.user_manager = UserManager(self)
|
||||
self.auth_manager = self._register_singleton(auth.AuthManager, auth.AuthManager(self.config))
|
||||
# Start the heartbeat process if configured and available (wait until
|
||||
# postfork if using uWSGI)
|
||||
if self.config.use_heartbeat:
|
||||
@@ -221,7 +243,7 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
|
||||
# Start the job manager
|
||||
from galaxy.jobs import manager
|
||||
self.job_manager = manager.JobManager(self)
|
||||
self.job_manager = self._register_singleton(manager.JobManager)
|
||||
self.application_stack.register_postfork_function(self.job_manager.start)
|
||||
self.proxy_manager = ProxyManager(self.config)
|
||||
|
||||
@@ -229,7 +251,7 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
# Must be initialized after job_config.
|
||||
self.workflow_scheduling_manager = scheduling_manager.WorkflowSchedulingManager(self)
|
||||
|
||||
self.trs_proxy = TrsProxy(self.config)
|
||||
self.trs_proxy = self._register_singleton(TrsProxy, TrsProxy(self.config))
|
||||
# Must be initialized after any component that might make use of stack messaging is configured. Alternatively if
|
||||
# it becomes more commonly needed we could create a prefork function registration method like we do with
|
||||
# postfork functions.
|
||||
@@ -341,7 +363,7 @@ class UniverseApplication(StructuredApp, config.ConfiguresGalaxyMixin):
|
||||
self.trace_logger = None
|
||||
|
||||
@property
|
||||
def is_job_handler(self):
|
||||
def is_job_handler(self) -> bool:
|
||||
return (self.config.track_jobs_in_database and self.job_config.is_handler) or not self.config.track_jobs_in_database
|
||||
|
||||
|
||||
|
||||
@@ -12,10 +12,9 @@ log = logging.getLogger(__name__)
|
||||
|
||||
class AuthManager:
|
||||
|
||||
def __init__(self, app):
|
||||
self.__app = app
|
||||
self.redact_username_in_logs = app.config.redact_username_in_logs
|
||||
self.authenticators = get_authenticators(app.config.auth_config_file, app.config.is_set('auth_config_file'))
|
||||
def __init__(self, config):
|
||||
self.redact_username_in_logs = config.redact_username_in_logs
|
||||
self.authenticators = get_authenticators(config.auth_config_file, config.is_set('auth_config_file'))
|
||||
|
||||
def check_registration_allowed(self, email, username, password):
|
||||
"""Checks if the provided email/username is allowed to register."""
|
||||
|
||||
@@ -816,7 +816,7 @@ class GalaxyAppConfiguration(BaseAppConfiguration, CommonConfigurationMixin):
|
||||
# InteractiveTools propagator mapping file
|
||||
self.interactivetools_map = self._in_root_dir(kwargs.get("interactivetools_map", self._in_data_dir("interactivetools_map.sqlite")))
|
||||
self.interactivetools_prefix = kwargs.get("interactivetools_prefix", "interactivetool")
|
||||
self.interactivetools_proxy_host = kwargs.get("interactivetool_proxy_host", None)
|
||||
self.interactivetools_proxy_host = kwargs.get("interactivetools_proxy_host", None)
|
||||
|
||||
self.containers_conf = parse_containers_config(self.containers_config_file)
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import logging
|
||||
from os.path import dirname
|
||||
|
||||
from galaxy.queue_worker import job_rule_modules
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.tools.toolbox.watcher import (
|
||||
get_tool_conf_watcher,
|
||||
get_tool_watcher,
|
||||
@@ -14,7 +15,7 @@ log = logging.getLogger(__name__)
|
||||
class ConfigWatchers:
|
||||
"""Contains ToolConfWatcher, ToolWatcher and ToolDataWatcher objects."""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
self.active = False
|
||||
# ToolConfWatcher objects will watch the tool_cache if the tool_cache is passed into get_tool_conf_watcher.
|
||||
|
||||
@@ -477,9 +477,9 @@ class Bed(Interval):
|
||||
if not get_headers(file_prefix, '\t', comment_designator='#', count=1):
|
||||
return False
|
||||
try:
|
||||
headers = iter_headers(file_prefix, '\t', comment_designator='#')
|
||||
for hdr in headers:
|
||||
if hdr[0] == '':
|
||||
found_valid_lines = False
|
||||
for hdr in iter_headers(file_prefix, '\t', comment_designator='#'):
|
||||
if not hdr or hdr == ['']:
|
||||
continue
|
||||
if len(hdr) < 3 or len(hdr) > 12:
|
||||
return False
|
||||
@@ -542,7 +542,8 @@ class Bed(Interval):
|
||||
return False
|
||||
if len(block_sizes) != block_count or len(block_starts) != block_count:
|
||||
return False
|
||||
return True
|
||||
found_valid_lines = True
|
||||
return found_valid_lines
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
@@ -818,28 +819,33 @@ class Gff(Tabular, _RemoteCallMixin):
|
||||
if len(get_headers(file_prefix, '\t', count=2)) < 2:
|
||||
return False
|
||||
try:
|
||||
headers = iter_headers(file_prefix, '\t')
|
||||
for hdr in headers:
|
||||
if hdr and hdr[0].startswith('##gff-version') and hdr[0].find('2') < 0:
|
||||
found_valid_lines = False
|
||||
for hdr in iter_headers(file_prefix, '\t'):
|
||||
if not hdr or hdr == ['']:
|
||||
continue
|
||||
if hdr[0].startswith('##gff-version') and hdr[0].find('2') < 0:
|
||||
return False
|
||||
if hdr and hdr[0] and not hdr[0].startswith('#'):
|
||||
if len(hdr) != 9:
|
||||
return False
|
||||
# The gff-version header comment may have been stripped, so inspect the data
|
||||
if hdr[0].startswith('#'):
|
||||
continue
|
||||
if len(hdr) != 9:
|
||||
return False
|
||||
try:
|
||||
int(hdr[3])
|
||||
int(hdr[4])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
int(hdr[3])
|
||||
int(hdr[4])
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[6] not in data.valid_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff_frame:
|
||||
return False
|
||||
return True
|
||||
if hdr[6] not in data.valid_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff_frame:
|
||||
return False
|
||||
found_valid_lines = True
|
||||
return found_valid_lines
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
@@ -953,37 +959,41 @@ class Gff3(Gff):
|
||||
if len(get_headers(file_prefix, '\t', count=2)) < 2:
|
||||
return False
|
||||
try:
|
||||
headers = iter_headers(file_prefix, '\t')
|
||||
for hdr in headers:
|
||||
if hdr and hdr[0].startswith('##gff-version') and hdr[0].find('3') >= 0:
|
||||
found_valid_lines = False
|
||||
for hdr in iter_headers(file_prefix, '\t'):
|
||||
if not hdr or hdr == ['']:
|
||||
continue
|
||||
if hdr[0].startswith('##gff-version') and hdr[0].find('3') >= 0:
|
||||
return True
|
||||
elif hdr and hdr[0].startswith('##gff-version') and hdr[0].find('3') < 0:
|
||||
elif hdr[0].startswith('##gff-version') and hdr[0].find('3') < 0:
|
||||
return False
|
||||
# Header comments may have been stripped, so inspect the data
|
||||
if hdr and hdr[0] and not hdr[0].startswith('#'):
|
||||
if len(hdr) != 9:
|
||||
# The gff-version header comment may have been stripped, so inspect the data
|
||||
if hdr[0].startswith('#'):
|
||||
continue
|
||||
if len(hdr) != 9:
|
||||
return False
|
||||
try:
|
||||
int(hdr[3])
|
||||
except Exception:
|
||||
if hdr[3] != '.':
|
||||
return False
|
||||
try:
|
||||
int(hdr[4])
|
||||
except Exception:
|
||||
if hdr[4] != '.':
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
int(hdr[3])
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
if hdr[3] != '.':
|
||||
return False
|
||||
try:
|
||||
int(hdr[4])
|
||||
except Exception:
|
||||
if hdr[4] != '.':
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[6] not in self.valid_gff3_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff3_phase:
|
||||
return False
|
||||
parse_gff3_attributes(hdr[8])
|
||||
return True
|
||||
if hdr[6] not in self.valid_gff3_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff3_phase:
|
||||
return False
|
||||
parse_gff3_attributes(hdr[8])
|
||||
found_valid_lines = True
|
||||
return found_valid_lines
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
@@ -1031,34 +1041,38 @@ class Gtf(Gff):
|
||||
if len(get_headers(file_prefix, '\t', count=2)) < 2:
|
||||
return False
|
||||
try:
|
||||
headers = iter_headers(file_prefix, '\t')
|
||||
for hdr in headers:
|
||||
if hdr and hdr[0].startswith('##gff-version') and hdr[0].find('2') < 0:
|
||||
found_valid_lines = False
|
||||
for hdr in iter_headers(file_prefix, '\t'):
|
||||
if not hdr or hdr == ['']:
|
||||
continue
|
||||
if hdr[0].startswith('##gff-version') and hdr[0].find('2') < 0:
|
||||
return False
|
||||
if hdr and hdr[0] and not hdr[0].startswith('#'):
|
||||
if len(hdr) != 9:
|
||||
return False
|
||||
# The gff-version header comment may have been stripped, so inspect the data
|
||||
if hdr[0].startswith('#'):
|
||||
continue
|
||||
if len(hdr) != 9:
|
||||
return False
|
||||
try:
|
||||
int(hdr[3])
|
||||
int(hdr[4])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
int(hdr[3])
|
||||
int(hdr[4])
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[5] != '.':
|
||||
try:
|
||||
float(hdr[5])
|
||||
except Exception:
|
||||
return False
|
||||
if hdr[6] not in data.valid_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff_frame:
|
||||
return False
|
||||
|
||||
# Check attributes for gene_id (transcript_id is also mandatory
|
||||
# but not for genes)
|
||||
attributes = parse_gff_attributes(hdr[8])
|
||||
if 'gene_id' not in attributes:
|
||||
return False
|
||||
return True
|
||||
if hdr[6] not in data.valid_strand:
|
||||
return False
|
||||
if hdr[7] not in self.valid_gff_frame:
|
||||
return False
|
||||
# Check attributes for gene_id (transcript_id is also mandatory
|
||||
# but not for genes)
|
||||
attributes = parse_gff_attributes(hdr[8])
|
||||
if 'gene_id' not in attributes:
|
||||
return False
|
||||
found_valid_lines = True
|
||||
return found_valid_lines
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
@@ -235,24 +235,22 @@ def is_column_based(fname_or_file_prefix, sep='\t', skip=0):
|
||||
return False
|
||||
|
||||
try:
|
||||
headers = get_headers(fname_or_file_prefix, sep)
|
||||
headers = get_headers(fname_or_file_prefix, sep, comment_designator='#')[skip:]
|
||||
except UnicodeDecodeError:
|
||||
return False
|
||||
count = 0
|
||||
if not headers:
|
||||
return False
|
||||
for hdr in headers[skip:]:
|
||||
if hdr and hdr[0] and not hdr[0].startswith('#'):
|
||||
if len(hdr) > 1:
|
||||
for hdr in headers:
|
||||
if hdr and hdr != ['']:
|
||||
if count:
|
||||
if len(hdr) != count:
|
||||
return False
|
||||
else:
|
||||
count = len(hdr)
|
||||
break
|
||||
if count < 2:
|
||||
return False
|
||||
for hdr in headers[skip:]:
|
||||
if hdr and hdr[0] and not hdr[0].startswith('#'):
|
||||
if len(hdr) != count:
|
||||
return False
|
||||
return True
|
||||
if count < 2:
|
||||
return False
|
||||
return count >= 2
|
||||
|
||||
|
||||
def guess_ext(fname, sniff_order, is_binary=False):
|
||||
@@ -303,13 +301,13 @@ def guess_ext(fname, sniff_order, is_binary=False):
|
||||
>>> guess_ext(fname, sniff_order)
|
||||
'gff3'
|
||||
>>> fname = get_test_fname('2.txt')
|
||||
>>> guess_ext(fname, sniff_order) # 2.txt
|
||||
>>> guess_ext(fname, sniff_order)
|
||||
'txt'
|
||||
>>> fname = get_test_fname('2.tabular')
|
||||
>>> guess_ext(fname, sniff_order)
|
||||
'tabular'
|
||||
>>> fname = get_test_fname('3.txt')
|
||||
>>> guess_ext(fname, sniff_order) # 3.txt
|
||||
>>> guess_ext(fname, sniff_order)
|
||||
'txt'
|
||||
>>> fname = get_test_fname('test_tab1.tabular')
|
||||
>>> guess_ext(fname, sniff_order)
|
||||
@@ -454,6 +452,9 @@ def guess_ext(fname, sniff_order, is_binary=False):
|
||||
>>> fname = get_test_fname('1imzml')
|
||||
>>> guess_ext(fname, sniff_order) # This test case is ensuring doesn't throw exception, actual value could change if non-utf encoding handling improves.
|
||||
'data'
|
||||
>>> fname = get_test_fname('too_many_comments_gff3.tabular')
|
||||
>>> guess_ext(fname, sniff_order) # It's a VCF but is sniffed as tabular because of the limit on the number of header lines we read
|
||||
'tabular'
|
||||
"""
|
||||
file_prefix = FilePrefix(fname)
|
||||
file_ext = run_sniffers_raw(file_prefix, sniff_order, is_binary)
|
||||
|
||||
@@ -744,13 +744,12 @@ class BaseVcf(Tabular):
|
||||
|
||||
def set_meta(self, dataset, **kwd):
|
||||
super().set_meta(dataset, **kwd)
|
||||
source = open(dataset.file_name)
|
||||
|
||||
# Skip comments.
|
||||
line = None
|
||||
for line in source:
|
||||
if not line.startswith('##'):
|
||||
break
|
||||
with compression_utils.get_fileobj(dataset.file_name) as fh:
|
||||
# Skip comments.
|
||||
for line in fh:
|
||||
if not line.startswith('##'):
|
||||
break
|
||||
|
||||
if line and line.startswith('#'):
|
||||
# Found header line, get sample names.
|
||||
@@ -816,7 +815,7 @@ class VcfGz(BaseVcf, binary.Binary):
|
||||
return binascii.hexlify(last28) == b'1f8b08040000000000ff0600424302001b0003000000000000000000'
|
||||
|
||||
def set_meta(self, dataset, **kwd):
|
||||
super(BaseVcf, self).set_meta(dataset, **kwd)
|
||||
super().set_meta(dataset, **kwd)
|
||||
""" Creates the index for the VCF file. """
|
||||
# These metadata values are not accessible by users, always overwrite
|
||||
index_file = dataset.metadata.tabix_index
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
ctgA est match 5410 7503 . - . ID=EST:agt830.3;Target=agt830.3+1+595
|
||||
ctgA est HSP 7000 7503 . - . Parent=EST:agt830.3;Target=agt830.3+1+504
|
||||
ctgA est HSP 5410 5500 . ? . Parent=EST:agt830.3;Target=agt830.3+505+595
|
||||
@@ -219,6 +219,14 @@ class ConditionalDependencies:
|
||||
def check_fs_webdavfs(self):
|
||||
return 'webdav' in self.file_sources
|
||||
|
||||
def check_fs_s3fs(self):
|
||||
# pyfilesystem plugin access to s3
|
||||
return 's3' in self.file_sources
|
||||
|
||||
def check_s3fs(self):
|
||||
# use s3fs directly (skipping pyfilesystem) for direct access to more options
|
||||
return 's3fs' in self.file_sources
|
||||
|
||||
def check_watchdog(self):
|
||||
install_set = {'auto', 'True', 'true', 'polling'}
|
||||
return (self.config['watch_tools'] in install_set
|
||||
|
||||
@@ -13,8 +13,11 @@ python-ldap==3.2.0
|
||||
python-pam
|
||||
galaxycloudrunner
|
||||
|
||||
fs.webdavfs
|
||||
fs.dropboxfs
|
||||
# For file sources plugins
|
||||
fs.webdavfs # type: webdav
|
||||
fs.dropboxfs # type: dropbox
|
||||
fs-s3fs # type: s3
|
||||
s3fs # type: s3fs
|
||||
|
||||
# Chronos client
|
||||
chronos-python==1.2.1
|
||||
|
||||
@@ -38,6 +38,7 @@ cmd2==1.4.0; python_version >= "3.6"
|
||||
colorama==0.4.4; python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6") or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.5.0" and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6")
|
||||
coloredlogs==15.0; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4" or python_version >= "3.6" and python_version < "4" and python_full_version >= "3.5.0"
|
||||
commonmark==0.9.1
|
||||
contextvars==2.4; python_version >= "3.6" and python_version < "3.7"
|
||||
coverage==5.3.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version < "4"
|
||||
cryptography==3.3.1; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
|
||||
cwltool==3.0.20201109103151; python_version >= "3.6" and python_version < "4"
|
||||
@@ -68,6 +69,7 @@ gxformat2==0.15.0
|
||||
h11==0.12.0; python_version >= "3.6"
|
||||
h5py==3.1.0; python_version >= "3.6"
|
||||
httplib2==0.18.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.4.0"
|
||||
httpx==0.16.1
|
||||
humanfriendly==9.1; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4" or python_version >= "3.6" and python_version < "4" and python_full_version >= "3.5.0"
|
||||
idna==2.10; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version < "4"
|
||||
imagesize==1.2.0; python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.5"
|
||||
@@ -83,6 +85,7 @@ jsonpatch==1.28; python_version >= "3.6" and python_full_version < "3.0.0" or py
|
||||
jsonpointer==2.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
|
||||
keystoneauth1==4.3.0; python_version >= "3.6"
|
||||
kombu==5.0.2; python_version >= "3.6"
|
||||
lagom==1.0.0; python_version >= "3.6" and python_version < "4.0"
|
||||
lockfile==0.12.2; python_version >= "3.6" and python_version < "4"
|
||||
lxml==4.6.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
|
||||
mako==1.1.4; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0")
|
||||
@@ -148,6 +151,7 @@ pyparsing==2.4.7; (python_version >= "2.6" and python_full_version < "3.0.0") or
|
||||
pyperclip==1.8.1; python_version >= "3.6"
|
||||
pyreadline==2.1; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4" and sys_platform == "win32" or python_version >= "3.6" and python_version < "4" and python_full_version >= "3.5.0" and sys_platform == "win32"
|
||||
pysam==0.16.0.1
|
||||
pytest-asyncio==0.14.0
|
||||
pytest-cov==2.11.1; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
|
||||
pytest-html==3.1.1; python_version >= "3.6"
|
||||
pytest-metadata==1.11.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
|
||||
@@ -208,6 +212,7 @@ sqlalchemy==1.3.22; (python_version >= "2.7" and python_full_version < "3.0.0")
|
||||
sqlitedict==1.7.0
|
||||
sqlparse==0.4.1; python_version >= "3.5"
|
||||
starlette==0.13.6; python_version >= "3.6" and python_version < "4.0"
|
||||
starlette-context==0.3.1; python_version >= "3.7"
|
||||
stevedore==3.3.0; python_version >= "3.6"
|
||||
svgwrite==1.4.1; python_version >= "3.6"
|
||||
tempita==0.5.2
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
flake8
|
||||
flake8-bugbear
|
||||
flake8-import-order
|
||||
mypy
|
||||
mypy==0.790
|
||||
|
||||
@@ -35,6 +35,7 @@ cloudbridge==2.1.0
|
||||
cmd2==1.4.0; python_version >= "3.6"
|
||||
colorama==0.4.4; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
|
||||
coloredlogs==15.0; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4" or python_version >= "3.6" and python_version < "4" and python_full_version >= "3.5.0"
|
||||
contextvars==2.4; python_version >= "3.6" and python_version < "3.7"
|
||||
cryptography==3.3.1; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
|
||||
cwltool==3.0.20201109103151; python_version >= "3.6" and python_version < "4"
|
||||
dataclasses==0.8; python_version >= "3.6" and python_version < "3.7"
|
||||
@@ -74,6 +75,7 @@ jsonpatch==1.28; python_version >= "3.6" and python_full_version < "3.0.0" or py
|
||||
jsonpointer==2.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
|
||||
keystoneauth1==4.3.0; python_version >= "3.6"
|
||||
kombu==5.0.2; python_version >= "3.6"
|
||||
lagom==1.0.0; python_version >= "3.6" and python_version < "4.0"
|
||||
lockfile==0.12.2; python_version >= "3.6" and python_version < "4"
|
||||
lxml==4.6.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
|
||||
mako==1.1.4; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0")
|
||||
@@ -168,6 +170,7 @@ sqlalchemy==1.3.22; (python_version >= "2.7" and python_full_version < "3.0.0")
|
||||
sqlitedict==1.7.0
|
||||
sqlparse==0.4.1; python_version >= "3.5"
|
||||
starlette==0.13.6; python_version >= "3.6" and python_version < "4.0"
|
||||
starlette-context==0.3.1; python_version >= "3.7"
|
||||
stevedore==3.3.0; python_version >= "3.6"
|
||||
svgwrite==1.4.1; python_version >= "3.6"
|
||||
tempita==0.5.2
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
"""Dependency injection framework for Galaxy-type apps."""
|
||||
from typing import Optional, Type, TypeVar
|
||||
|
||||
from lagom import Container as LagomContainer
|
||||
from lagom.exceptions import UnresolvableType
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
class Container(LagomContainer):
|
||||
"""Abstraction around lagom to provide a dependency injection context.
|
||||
|
||||
Abstractions used by Galaxy should come through this interface so we can swap
|
||||
out the backend as needed. For instance https://punq.readthedocs.io/en/latest/
|
||||
containers look very nice and would allow us to also inject by name (e.g. for
|
||||
config variables for instance).
|
||||
"""
|
||||
|
||||
def _register_singleton(self, dep_type: Type[T], instance: Optional[T] = None) -> T:
|
||||
if instance is None:
|
||||
# create an instance from the context and register it as a singleton
|
||||
instance = self[dep_type]
|
||||
self[dep_type] = instance
|
||||
return self[dep_type]
|
||||
|
||||
def resolve_or_none(self, dep_type: Type[T]) -> Optional[T]:
|
||||
"""Resolve the dependent type or just return None.
|
||||
|
||||
If resolution is impossible assume caller has a backup plan for
|
||||
constructing the desired object. Used to construct controllers that
|
||||
may or may not be resolvable (some have upgraded but legacy framework still
|
||||
works).
|
||||
"""
|
||||
try:
|
||||
return self[dep_type]
|
||||
except UnresolvableType:
|
||||
return None
|
||||
@@ -0,0 +1,83 @@
|
||||
import functools
|
||||
import os
|
||||
|
||||
try:
|
||||
import s3fs
|
||||
except ImportError:
|
||||
s3fs = None
|
||||
|
||||
from ..sources import BaseFilesSource
|
||||
|
||||
DEFAULT_ENFORCE_SYMLINK_SECURITY = True
|
||||
DEFAULT_DELETE_ON_REALIZE = False
|
||||
|
||||
|
||||
class S3FsFilesSource(BaseFilesSource):
|
||||
plugin_type = 's3fs'
|
||||
|
||||
def __init__(self, **kwd):
|
||||
if s3fs is None:
|
||||
raise Exception("Package s3fs unavailable but required for this file source plugin.")
|
||||
props = self._parse_common_config_opts(kwd)
|
||||
self._bucket = props.pop("bucket")
|
||||
assert self._bucket
|
||||
self._props = props
|
||||
|
||||
def list(self, path="/", recursive=True, user_context=None):
|
||||
fs = self._open_fs(user_context=user_context)
|
||||
if recursive:
|
||||
res = []
|
||||
bucket_path = self._bucket_path(path)
|
||||
for p, dirs, files in fs.walk(bucket_path, detail=True):
|
||||
to_dict = functools.partial(self._resource_info_to_dict, p)
|
||||
res.extend(map(to_dict, dirs.values()))
|
||||
res.extend(map(to_dict, files.values()))
|
||||
return res
|
||||
else:
|
||||
bucket_path = self._bucket_path(path)
|
||||
res = fs.ls(bucket_path, detail=True)
|
||||
to_dict = functools.partial(self._resource_info_to_dict, path)
|
||||
return list(map(to_dict, res))
|
||||
|
||||
def realize_to(self, source_path, native_path, user_context=None):
|
||||
bucket_path = self._bucket_path(source_path)
|
||||
self._open_fs(user_context=user_context).download(bucket_path, native_path)
|
||||
|
||||
def _write_from(self, target_path, native_path, user_context=None):
|
||||
raise NotImplementedError()
|
||||
|
||||
def _bucket_path(self, path):
|
||||
if not path.startswith("/"):
|
||||
path = f"/{path}"
|
||||
return f"{self._bucket}{path}"
|
||||
|
||||
def _open_fs(self, user_context=None):
|
||||
fs = s3fs.S3FileSystem(**self._props)
|
||||
return fs
|
||||
|
||||
def _resource_info_to_dict(self, dir_path, resource_info):
|
||||
name = os.path.basename(resource_info["name"])
|
||||
path = os.path.join(dir_path, name)
|
||||
uri = self.uri_from_path(path)
|
||||
if resource_info["type"] == "directory":
|
||||
return {"class": "Directory", "name": name, "uri": uri, "path": path}
|
||||
else:
|
||||
return {
|
||||
"class": "File",
|
||||
"name": name,
|
||||
"size": resource_info["size"],
|
||||
# should this be mtime...
|
||||
"ctime": self.to_dict_time(resource_info["LastModified"]),
|
||||
"uri": uri,
|
||||
"path": path,
|
||||
}
|
||||
|
||||
def _serialization_props(self, user_context=None):
|
||||
effective_props = {}
|
||||
for key, val in self._props.items():
|
||||
effective_props[key] = self._evaluate_prop(val, user_context=user_context)
|
||||
effective_props["bucket"] = self._bucket
|
||||
return effective_props
|
||||
|
||||
|
||||
__all__ = ('S3FsFilesSource',)
|
||||
@@ -18,6 +18,7 @@ from abc import (
|
||||
abstractmethod,
|
||||
)
|
||||
from json import loads
|
||||
from typing import Any, Dict, List
|
||||
|
||||
import packaging.version
|
||||
import yaml
|
||||
@@ -56,6 +57,7 @@ from galaxy.jobs.runners import BaseJobRunner, JobState
|
||||
from galaxy.metadata import get_metadata_compute_strategy
|
||||
from galaxy.model import store
|
||||
from galaxy.objectstore import ObjectStorePopulator
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.tool_util.deps import requirements
|
||||
from galaxy.tool_util.output_checker import (
|
||||
check_output,
|
||||
@@ -277,6 +279,13 @@ class JobConfiguration(ConfiguresHandlers):
|
||||
|
||||
These features are configured in the job configuration, by default, ``job_conf.xml``
|
||||
"""
|
||||
runner_plugins: List[dict]
|
||||
handlers: dict
|
||||
handler_runner_plugins: Dict[str, str]
|
||||
tools: Dict[str, list]
|
||||
resource_groups: Dict[str, list]
|
||||
destinations: Dict[str, tuple]
|
||||
resource_parameters: Dict[str, Any]
|
||||
DEFAULT_BASE_HANDLER_POOLS = ('job-handlers',)
|
||||
|
||||
DEFAULT_NWORKERS = 4
|
||||
@@ -292,7 +301,7 @@ class JobConfiguration(ConfiguresHandlers):
|
||||
<when value="yes"/>
|
||||
</conditional>"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
"""Parse the job configuration XML.
|
||||
"""
|
||||
self.app = app
|
||||
@@ -306,7 +315,6 @@ class JobConfiguration(ConfiguresHandlers):
|
||||
self.handler_max_grab = None
|
||||
self.handler_ready_window_size = None
|
||||
self.destinations = {}
|
||||
self.destination_tags = {}
|
||||
self.default_destination_id = None
|
||||
self.tools = {}
|
||||
self.resource_groups = {}
|
||||
|
||||
@@ -40,7 +40,16 @@ JOB_WAIT, JOB_ERROR, JOB_INPUT_ERROR, JOB_INPUT_DELETED, JOB_READY, JOB_DELETED,
|
||||
DEFAULT_JOB_PUT_FAILURE_MESSAGE = 'Unable to run job due to a misconfiguration of the Galaxy job running system. Please contact a site administrator.'
|
||||
|
||||
|
||||
class JobHandler:
|
||||
class JobHandlerI:
|
||||
|
||||
def start(self):
|
||||
pass
|
||||
|
||||
def shutdown(self):
|
||||
pass
|
||||
|
||||
|
||||
class JobHandler(JobHandlerI):
|
||||
"""
|
||||
Handle the preparation, running, tracking, and finishing of jobs
|
||||
"""
|
||||
|
||||
@@ -9,6 +9,7 @@ from sqlalchemy.sql.expression import null
|
||||
from galaxy.exceptions import HandlerAssignmentError, ToolExecutionError
|
||||
from galaxy.jobs import handler, NoopQueue
|
||||
from galaxy.model import Job
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.web_stack.message import JobHandlerMessage
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -18,7 +19,9 @@ class JobManager:
|
||||
"""
|
||||
Highest level interface to job management.
|
||||
"""
|
||||
def __init__(self, app):
|
||||
job_handler: handler.JobHandlerI
|
||||
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
self.job_lock = False
|
||||
if self.app.is_job_handler:
|
||||
@@ -110,7 +113,7 @@ class NoopManager:
|
||||
pass
|
||||
|
||||
|
||||
class NoopHandler:
|
||||
class NoopHandler(handler.JobHandlerI):
|
||||
"""
|
||||
Implements the JobHandler interface but does nothing
|
||||
"""
|
||||
|
||||
@@ -153,6 +153,7 @@ class SlurmJobRunner(DRMAAJobRunner):
|
||||
self.work_queue.put((self.fail_job, ajs))
|
||||
return
|
||||
if drmaa_state == self.drmaa_job_states.DONE:
|
||||
ajs.job_wrapper.reclaim_ownership()
|
||||
with open(ajs.error_file) as rfh:
|
||||
_remove_spurious_top_lines(rfh, ajs)
|
||||
with open(ajs.error_file, 'r+') as f:
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
from galaxy.structured_app import BasicApp
|
||||
|
||||
|
||||
class ApiKeyManager:
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: BasicApp):
|
||||
self.app = app
|
||||
|
||||
def create_api_key(self, user):
|
||||
def create_api_key(self, user) -> str:
|
||||
guid = self.app.security.get_new_guid()
|
||||
new_key = self.app.model.APIKeys()
|
||||
new_key.user_id = user.id
|
||||
@@ -13,7 +16,7 @@ class ApiKeyManager:
|
||||
sa_session.flush()
|
||||
return guid
|
||||
|
||||
def get_or_create_api_key(self, user):
|
||||
def get_or_create_api_key(self, user) -> str:
|
||||
# Logic Galaxy has always used - but it would appear to have a race
|
||||
# condition. Worth fixing? Would kind of need a message queue to fix
|
||||
# in multiple process mode.
|
||||
|
||||
+19
-12
@@ -28,14 +28,16 @@ attribute change to a model object.
|
||||
import datetime
|
||||
import logging
|
||||
import re
|
||||
from typing import Type
|
||||
from typing import Callable, Dict, List, Optional, Set, Type
|
||||
|
||||
import routes
|
||||
import sqlalchemy
|
||||
from sqlalchemy.orm.scoping import scoped_session
|
||||
|
||||
from galaxy import exceptions
|
||||
from galaxy import model
|
||||
from galaxy.model import tool_shed_install
|
||||
from galaxy.structured_app import BasicApp, StructuredApp
|
||||
from galaxy.util import namedtuple
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -158,11 +160,12 @@ class ModelManager:
|
||||
"""
|
||||
model_class: type = object
|
||||
foreign_key_name: str
|
||||
app: BasicApp
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: BasicApp):
|
||||
self.app = app
|
||||
|
||||
def session(self):
|
||||
def session(self) -> scoped_session:
|
||||
return self.app.model.context
|
||||
|
||||
def _session_setattr(self, item, attr, val, fn=None, flush=True):
|
||||
@@ -485,7 +488,7 @@ class HasAModelManager:
|
||||
# examples where this doesn't really work are ConfigurationSerializer (no manager)
|
||||
# and contents (2 managers)
|
||||
|
||||
def __init__(self, app, manager=None, **kwargs):
|
||||
def __init__(self, app: StructuredApp, manager=None, **kwargs):
|
||||
self._manager = manager
|
||||
|
||||
@property
|
||||
@@ -494,7 +497,7 @@ class HasAModelManager:
|
||||
# PRECONDITION: assumes self.app is assigned elsewhere
|
||||
if not self._manager:
|
||||
# TODO: pass this serializer to it
|
||||
self._manager = self.model_manager_class(self.app)
|
||||
self._manager = self.app[self.model_manager_class]
|
||||
# this will error for unset model_manager_class'es
|
||||
return self._manager
|
||||
|
||||
@@ -536,8 +539,10 @@ class ModelSerializer(HasAModelManager):
|
||||
"""
|
||||
#: 'service' to use for getting urls - use class var to allow overriding when testing
|
||||
url_for = staticmethod(routes.url_for)
|
||||
default_view: Optional[str]
|
||||
views: Dict[str, List[str]]
|
||||
|
||||
def __init__(self, app, **kwargs):
|
||||
def __init__(self, app: StructuredApp, **kwargs):
|
||||
"""
|
||||
Set up serializer map, any additional serializable keys, and views here.
|
||||
"""
|
||||
@@ -548,9 +553,9 @@ class ModelSerializer(HasAModelManager):
|
||||
# this allows us to: 'mention' the key without adding the default serializer
|
||||
# TODO: we may want to eventually error if a key is requested
|
||||
# that is in neither serializable_keyset or serializers
|
||||
self.serializable_keyset = set()
|
||||
self.serializable_keyset: Set[str] = set()
|
||||
# a map of dictionary keys to the functions (often lambdas) that create the values for those keys
|
||||
self.serializers = {}
|
||||
self.serializers: Dict[str, Callable] = {}
|
||||
# add subclass serializers defined there
|
||||
self.add_serializers()
|
||||
# update the keyset by the serializers (removing the responsibility from subclasses)
|
||||
@@ -708,15 +713,15 @@ class ModelDeserializer(HasAModelManager):
|
||||
"""
|
||||
# TODO:?? a larger question is: which should be first? Deserialize then validate - or - validate then deserialize?
|
||||
|
||||
def __init__(self, app, validator=None, **kwargs):
|
||||
def __init__(self, app: StructuredApp, validator=None, **kwargs):
|
||||
"""
|
||||
Set up deserializers and validator.
|
||||
"""
|
||||
super().__init__(app, **kwargs)
|
||||
self.app = app
|
||||
|
||||
self.deserializers = {}
|
||||
self.deserializable_keyset = set()
|
||||
self.deserializers: Dict[str, Callable] = {}
|
||||
self.deserializable_keyset: Set[str] = set()
|
||||
self.add_deserializers()
|
||||
# a sub object that can validate incoming values
|
||||
self.validate = validator or ModelValidator(self.app)
|
||||
@@ -896,8 +901,10 @@ class ModelFilterParser(HasAModelManager):
|
||||
|
||||
model_class: type
|
||||
parsed_filter = parsed_filter
|
||||
orm_filter_parsers: Dict[str, Dict]
|
||||
fn_filter_parsers: Dict[str, Dict]
|
||||
|
||||
def __init__(self, app, **kwargs):
|
||||
def __init__(self, app: StructuredApp, **kwargs):
|
||||
"""
|
||||
Set up serializer map, any additional serializable keys, and views here.
|
||||
"""
|
||||
|
||||
@@ -5,12 +5,14 @@ import requests
|
||||
from beaker.cache import CacheManager
|
||||
from beaker.util import parse_cache_config_options
|
||||
|
||||
from galaxy.structured_app import BasicApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CitationsManager:
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: BasicApp) -> None:
|
||||
self.app = app
|
||||
self.doi_cache = DoiCache(app.config)
|
||||
|
||||
|
||||
@@ -50,9 +50,6 @@ class CloudManager(sharable.SharableModelManager):
|
||||
# setting this property.
|
||||
model_class = model.History
|
||||
|
||||
def __init__(self, app, *args, **kwargs):
|
||||
super().__init__(app, *args, **kwargs)
|
||||
|
||||
@staticmethod
|
||||
def configure_provider(provider, credentials):
|
||||
"""
|
||||
|
||||
@@ -8,20 +8,21 @@ from galaxy.exceptions import (
|
||||
MessageException,
|
||||
RequestParameterInvalidException
|
||||
)
|
||||
from galaxy.managers import (
|
||||
hdas,
|
||||
histories,
|
||||
lddas,
|
||||
)
|
||||
from galaxy.managers.collections_util import validate_input_element_identifiers
|
||||
from galaxy.model import tags
|
||||
from galaxy.model.dataset_collections import builder
|
||||
from galaxy.model.dataset_collections.matching import MatchingCollections
|
||||
from galaxy.model.dataset_collections.registry import DATASET_COLLECTION_TYPES_REGISTRY
|
||||
from galaxy.model.dataset_collections.type_description import COLLECTION_TYPE_DESCRIPTION_FACTORY
|
||||
from galaxy.model.mapping import GalaxyModelMapping
|
||||
from galaxy.model.tags import GalaxyTagHandler
|
||||
from galaxy.security.idencoding import IdEncodingHelper
|
||||
from galaxy.util import (
|
||||
validation
|
||||
)
|
||||
from .hdas import HDAManager
|
||||
from .histories import HistoryManager
|
||||
from .lddas import LDDAManager
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -36,16 +37,24 @@ class DatasetCollectionManager:
|
||||
"""
|
||||
ELEMENTS_UNINITIALIZED = object()
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(
|
||||
self,
|
||||
model: GalaxyModelMapping,
|
||||
security: IdEncodingHelper,
|
||||
hda_manager: HDAManager,
|
||||
history_manager: HistoryManager,
|
||||
tag_handler: GalaxyTagHandler,
|
||||
ldda_manager: LDDAManager,
|
||||
):
|
||||
self.type_registry = DATASET_COLLECTION_TYPES_REGISTRY
|
||||
self.collection_type_descriptions = COLLECTION_TYPE_DESCRIPTION_FACTORY
|
||||
self.model = app.model
|
||||
self.security = app.security
|
||||
self.model = model
|
||||
self.security = security
|
||||
|
||||
self.hda_manager = hdas.HDAManager(app)
|
||||
self.history_manager = histories.HistoryManager(app)
|
||||
self.tag_handler = tags.GalaxyTagHandler(app.model.context)
|
||||
self.ldda_manager = lddas.LDDAManager(app)
|
||||
self.hda_manager = hda_manager
|
||||
self.history_manager = history_manager
|
||||
self.tag_handler = tag_handler
|
||||
self.ldda_manager = ldda_manager
|
||||
|
||||
def precreate_dataset_collection_instance(self, trans, parent, name, structure, implicit_inputs=None, implicit_output_name=None, tags=None, completed_collection=None):
|
||||
# TODO: prebuild all required HIDs and send them in so no need to flush in between.
|
||||
@@ -136,7 +145,7 @@ class DatasetCollectionManager:
|
||||
|
||||
def _create_instance_for_collection(self, trans, parent, name, dataset_collection, implicit_output_name=None, implicit_inputs=None, tags=None, set_hid=True, flush=True):
|
||||
if isinstance(parent, model.History):
|
||||
dataset_collection_instance = self.model.HistoryDatasetCollectionAssociation(
|
||||
dataset_collection_instance = model.HistoryDatasetCollectionAssociation(
|
||||
collection=dataset_collection,
|
||||
name=name,
|
||||
)
|
||||
@@ -153,7 +162,7 @@ class DatasetCollectionManager:
|
||||
parent.add_dataset_collection(dataset_collection_instance)
|
||||
|
||||
elif isinstance(parent, model.LibraryFolder):
|
||||
dataset_collection_instance = self.model.LibraryDatasetCollectionAssociation(
|
||||
dataset_collection_instance = model.LibraryDatasetCollectionAssociation(
|
||||
collection=dataset_collection,
|
||||
folder=parent,
|
||||
name=name,
|
||||
|
||||
@@ -18,6 +18,7 @@ from galaxy.managers import (
|
||||
secured,
|
||||
users
|
||||
)
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util.checkers import check_binary
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -32,7 +33,7 @@ class DatasetManager(base.ModelManager, secured.AccessibleManagerMixin, deletabl
|
||||
|
||||
# TODO:?? get + error_if_uploading is common pattern, should upload check be worked into access/owed?
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.permissions = DatasetRBACPermissions(app)
|
||||
# needed for admin test
|
||||
@@ -142,11 +143,11 @@ class DatasetRBACPermissions:
|
||||
class DatasetSerializer(base.ModelSerializer, deletable.PurgableSerializerMixin):
|
||||
model_manager_class = DatasetManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp, user_manager: users.UserManager):
|
||||
super().__init__(app)
|
||||
self.dataset_manager = self.manager
|
||||
# needed for admin test
|
||||
self.user_manager = users.UserManager(app)
|
||||
self.user_manager = user_manager
|
||||
|
||||
self.default_view = 'summary'
|
||||
self.add_view('summary', [
|
||||
@@ -438,7 +439,7 @@ class _UnflattenedMetadataDatasetAssociationSerializer(base.ModelSerializer,
|
||||
deletable.PurgableSerializerMixin):
|
||||
|
||||
def __init__(self, app):
|
||||
self.dataset_serializer = DatasetSerializer(app)
|
||||
self.dataset_serializer = app[DatasetSerializer]
|
||||
super().__init__(app)
|
||||
|
||||
def add_serializers(self):
|
||||
|
||||
@@ -5,6 +5,7 @@ from typing import (
|
||||
)
|
||||
|
||||
from galaxy.datatypes.registry import Registry
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -12,7 +13,7 @@ log = logging.getLogger(__name__)
|
||||
class DisplayApplicationsManager:
|
||||
"""Interface/service object for sharing logic between controllers."""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self._app = app
|
||||
|
||||
@property
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
import logging
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
List,
|
||||
Optional,
|
||||
)
|
||||
|
||||
from galaxy import model
|
||||
from galaxy.app import StructuredApp
|
||||
from galaxy.exceptions import (
|
||||
ObjectNotFound,
|
||||
)
|
||||
from galaxy.managers.base import decode_id
|
||||
from galaxy.managers.context import ProvidesAppContext
|
||||
from galaxy.schema.fields import EncodedDatabaseIdField
|
||||
from galaxy.web import url_for
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GroupRolesManager:
|
||||
"""Interface/service object shared by controllers for interacting with group roles."""
|
||||
|
||||
def __init__(self, app: StructuredApp) -> None:
|
||||
self._app = app
|
||||
|
||||
def index(self, trans: ProvidesAppContext, group_id: EncodedDatabaseIdField) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Returns a collection roles associated with the given group.
|
||||
"""
|
||||
group = self._get_group(trans, group_id)
|
||||
rval = []
|
||||
for gra in group.roles:
|
||||
group_role = self._serialize_group_role(group_id, gra.role)
|
||||
rval.append(group_role)
|
||||
return rval
|
||||
|
||||
def show(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns information about a group role.
|
||||
"""
|
||||
role_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
role = self._get_role(trans, role_id)
|
||||
group_role = self._get_group_role(trans, group, role)
|
||||
if group_role is None:
|
||||
raise ObjectNotFound(f"Role {role.name} not in group {group.name}")
|
||||
|
||||
return self._serialize_group_role(group_id, role)
|
||||
|
||||
def update(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField):
|
||||
"""
|
||||
Adds a role to a group if it is not already associated.
|
||||
"""
|
||||
role_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
role = self._get_role(trans, role_id)
|
||||
group_role = self._get_group_role(trans, group, role)
|
||||
if group_role is None:
|
||||
self._add_role_to_group(trans, group, role)
|
||||
|
||||
return self._serialize_group_role(group_id, role)
|
||||
|
||||
def delete(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField):
|
||||
"""
|
||||
Removes a role from a group.
|
||||
"""
|
||||
role_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
role = self._get_role(trans, role_id)
|
||||
group_role = self._get_group_role(trans, group, role)
|
||||
if group_role is None:
|
||||
raise ObjectNotFound(f"Role {role.name} not in group {group.name}")
|
||||
self._remove_role_from_group(trans, group_role)
|
||||
return self._serialize_group_role(group_id, role)
|
||||
|
||||
def _get_group(self, trans: ProvidesAppContext, encoded_group_id: EncodedDatabaseIdField) -> Any:
|
||||
decoded_group_id = decode_id(self._app, encoded_group_id)
|
||||
group = trans.sa_session.query(model.Group).get(decoded_group_id)
|
||||
if group is None:
|
||||
raise ObjectNotFound(f"Group with id {encoded_group_id} was not found.")
|
||||
return group
|
||||
|
||||
def _get_role(self, trans: ProvidesAppContext, encoded_role_id: EncodedDatabaseIdField) -> model.Role:
|
||||
decoded_role_id = decode_id(self._app, encoded_role_id)
|
||||
role = trans.sa_session.query(model.Role).get(decoded_role_id)
|
||||
if role is None:
|
||||
raise ObjectNotFound(f"Role with id {encoded_role_id} was not found.")
|
||||
return role
|
||||
|
||||
def _get_group_role(self, trans: ProvidesAppContext, group: model.Group, role: model.Role) -> Optional[model.GroupRoleAssociation]:
|
||||
return trans.sa_session.query(model.GroupRoleAssociation).filter(
|
||||
model.GroupRoleAssociation.group == group,
|
||||
model.GroupRoleAssociation.role == role
|
||||
).one_or_none()
|
||||
|
||||
def _add_role_to_group(self, trans: ProvidesAppContext, group: model.Group, role: model.Role):
|
||||
gra = model.GroupRoleAssociation(group, role)
|
||||
trans.sa_session.add(gra)
|
||||
trans.sa_session.flush()
|
||||
|
||||
def _remove_role_from_group(self, trans: ProvidesAppContext, group_role: model.GroupRoleAssociation):
|
||||
trans.sa_session.delete(group_role)
|
||||
trans.sa_session.flush()
|
||||
|
||||
def _serialize_group_role(self, encoded_group_id: EncodedDatabaseIdField, role: model.Role):
|
||||
encoded_role_id = self._app.security.encode_id(role.id)
|
||||
return {
|
||||
"id": encoded_role_id,
|
||||
"name": role.name,
|
||||
"url": url_for('group_role', group_id=encoded_group_id, id=encoded_role_id)
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
import logging
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
List,
|
||||
Optional,
|
||||
)
|
||||
|
||||
from galaxy import model
|
||||
from galaxy.app import StructuredApp
|
||||
from galaxy.exceptions import (
|
||||
ObjectNotFound,
|
||||
)
|
||||
from galaxy.managers.base import decode_id
|
||||
from galaxy.managers.context import ProvidesAppContext
|
||||
from galaxy.schema.fields import EncodedDatabaseIdField
|
||||
from galaxy.web import url_for
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GroupUsersManager:
|
||||
"""Interface/service object shared by controllers for interacting with group users."""
|
||||
|
||||
def __init__(self, app: StructuredApp) -> None:
|
||||
self._app = app
|
||||
|
||||
def index(self, trans: ProvidesAppContext, group_id: EncodedDatabaseIdField) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Returns a collection (list) with some information about users associated with the given group.
|
||||
"""
|
||||
group = self._get_group(trans, group_id)
|
||||
rval = []
|
||||
for uga in group.users:
|
||||
group_user = self._serialize_group_user(group_id, uga.user)
|
||||
rval.append(group_user)
|
||||
return rval
|
||||
|
||||
def show(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField) -> Dict[str, Any]:
|
||||
"""
|
||||
Returns information about a group user.
|
||||
"""
|
||||
user_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
user = self._get_user(trans, user_id)
|
||||
group_user = self._get_group_user(trans, group, user)
|
||||
if group_user is None:
|
||||
raise ObjectNotFound(f"User {user.email} not in group {group.name}")
|
||||
|
||||
return self._serialize_group_user(group_id, user)
|
||||
|
||||
def update(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField):
|
||||
"""
|
||||
Adds a user to a group.
|
||||
"""
|
||||
user_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
user = self._get_user(trans, user_id)
|
||||
group_user = self._get_group_user(trans, group, user)
|
||||
if group_user is None:
|
||||
self._add_user_to_group(trans, group, user)
|
||||
|
||||
return self._serialize_group_user(group_id, user)
|
||||
|
||||
def delete(self, trans: ProvidesAppContext, id: EncodedDatabaseIdField, group_id: EncodedDatabaseIdField):
|
||||
"""
|
||||
Removes a user from a group.
|
||||
"""
|
||||
user_id = id
|
||||
group = self._get_group(trans, group_id)
|
||||
user = self._get_user(trans, user_id)
|
||||
group_user = self._get_group_user(trans, group, user)
|
||||
if group_user is None:
|
||||
raise ObjectNotFound(f"User {user.email} not in group {group.name}")
|
||||
self._remove_user_from_group(trans, group_user)
|
||||
return self._serialize_group_user(group_id, user)
|
||||
|
||||
def _get_group(self, trans: ProvidesAppContext, encoded_group_id: EncodedDatabaseIdField) -> Any:
|
||||
decoded_group_id = decode_id(self._app, encoded_group_id)
|
||||
group = trans.sa_session.query(model.Group).get(decoded_group_id)
|
||||
if group is None:
|
||||
raise ObjectNotFound(f"Group with id {encoded_group_id} was not found.")
|
||||
return group
|
||||
|
||||
def _get_user(self, trans: ProvidesAppContext, encoded_user_id: EncodedDatabaseIdField) -> model.User:
|
||||
decoded_user_id = decode_id(self._app, encoded_user_id)
|
||||
user = trans.sa_session.query(model.User).get(decoded_user_id)
|
||||
if user is None:
|
||||
raise ObjectNotFound(f"User with id {encoded_user_id} was not found.")
|
||||
return user
|
||||
|
||||
def _get_group_user(self, trans: ProvidesAppContext, group: model.Group, user: model.User) -> Optional[model.UserGroupAssociation]:
|
||||
return trans.sa_session.query(model.UserGroupAssociation).filter(
|
||||
model.UserGroupAssociation.user == user,
|
||||
model.UserGroupAssociation.group == group
|
||||
).one_or_none()
|
||||
|
||||
def _add_user_to_group(self, trans: ProvidesAppContext, group: model.Group, user: model.User):
|
||||
gra = model.UserGroupAssociation(user, group)
|
||||
trans.sa_session.add(gra)
|
||||
trans.sa_session.flush()
|
||||
|
||||
def _remove_user_from_group(self, trans: ProvidesAppContext, group_user: model.UserGroupAssociation):
|
||||
trans.sa_session.delete(group_user)
|
||||
trans.sa_session.flush()
|
||||
|
||||
def _serialize_group_user(self, encoded_group_id: EncodedDatabaseIdField, user: model.User):
|
||||
encoded_user_id = self._app.security.encode_id(user.id)
|
||||
return {
|
||||
"id": encoded_user_id,
|
||||
"email": user.email,
|
||||
"url": url_for('group_user', group_id=encoded_group_id, id=encoded_user_id)
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
List,
|
||||
)
|
||||
|
||||
from sqlalchemy import false
|
||||
|
||||
from galaxy import model
|
||||
from galaxy.app import StructuredApp
|
||||
from galaxy.exceptions import (
|
||||
Conflict,
|
||||
ObjectAttributeMissingException,
|
||||
ObjectNotFound,
|
||||
)
|
||||
from galaxy.managers.base import decode_id
|
||||
from galaxy.managers.context import ProvidesAppContext
|
||||
from galaxy.schema.fields import EncodedDatabaseIdField
|
||||
from galaxy.web import url_for
|
||||
|
||||
|
||||
class GroupsManager:
|
||||
"""Interface/service object shared by controllers for interacting with groups."""
|
||||
|
||||
def __init__(self, app: StructuredApp) -> None:
|
||||
self._app = app
|
||||
|
||||
def index(self, trans: ProvidesAppContext):
|
||||
"""
|
||||
Displays a collection (list) of groups.
|
||||
"""
|
||||
rval = []
|
||||
for group in trans.sa_session.query(model.Group).filter(model.Group.table.c.deleted == false()):
|
||||
item = group.to_dict(value_mapper={'id': trans.security.encode_id})
|
||||
encoded_id = trans.security.encode_id(group.id)
|
||||
item['url'] = url_for('group', id=encoded_id)
|
||||
rval.append(item)
|
||||
return rval
|
||||
|
||||
def create(self, trans: ProvidesAppContext, payload: Dict[str, Any]):
|
||||
"""
|
||||
Creates a new group.
|
||||
"""
|
||||
name = payload.get('name', None)
|
||||
if name is None:
|
||||
raise ObjectAttributeMissingException("Missing required name")
|
||||
self._check_duplicated_group_name(trans, name)
|
||||
|
||||
group = model.Group(name=name)
|
||||
trans.sa_session.add(group)
|
||||
encoded_user_ids = payload.get('user_ids', [])
|
||||
users = self._get_users_by_encoded_ids(trans, encoded_user_ids)
|
||||
encoded_role_ids = payload.get('role_ids', [])
|
||||
roles = self._get_roles_by_encoded_ids(trans, encoded_role_ids)
|
||||
trans.app.security_agent.set_entity_group_associations(groups=[group], roles=roles, users=users)
|
||||
trans.sa_session.flush()
|
||||
|
||||
encoded_id = trans.security.encode_id(group.id)
|
||||
item = group.to_dict(view='element', value_mapper={'id': trans.security.encode_id})
|
||||
item['url'] = url_for('group', id=encoded_id)
|
||||
return [item]
|
||||
|
||||
def show(self, trans: ProvidesAppContext, encoded_id: EncodedDatabaseIdField):
|
||||
"""
|
||||
Displays information about a group.
|
||||
"""
|
||||
group = self._get_group(trans, encoded_id)
|
||||
item = group.to_dict(view='element', value_mapper={'id': trans.security.encode_id})
|
||||
item['url'] = url_for('group', id=encoded_id)
|
||||
item['users_url'] = url_for('group_users', group_id=encoded_id)
|
||||
item['roles_url'] = url_for('group_roles', group_id=encoded_id)
|
||||
return item
|
||||
|
||||
def update(self, trans: ProvidesAppContext, encoded_id: EncodedDatabaseIdField, payload: Dict[str, Any]):
|
||||
"""
|
||||
Modifies a group.
|
||||
"""
|
||||
group = self._get_group(trans, encoded_id)
|
||||
name = payload.get('name', None)
|
||||
if name:
|
||||
self._check_duplicated_group_name(trans, name)
|
||||
group.name = name
|
||||
trans.sa_session.add(group)
|
||||
encoded_user_ids = payload.get('user_ids', [])
|
||||
users = self._get_users_by_encoded_ids(trans, encoded_user_ids)
|
||||
encoded_role_ids = payload.get('role_ids', [])
|
||||
roles = self._get_roles_by_encoded_ids(trans, encoded_role_ids)
|
||||
trans.app.security_agent.set_entity_group_associations(groups=[group], roles=roles, users=users, delete_existing_assocs=False)
|
||||
trans.sa_session.flush()
|
||||
|
||||
def _decode_id(self, encoded_id: EncodedDatabaseIdField) -> int:
|
||||
return decode_id(self._app, encoded_id)
|
||||
|
||||
def _decode_ids(self, encoded_ids: List[EncodedDatabaseIdField]) -> List[int]:
|
||||
return [self._decode_id(encoded_id) for encoded_id in encoded_ids]
|
||||
|
||||
def _check_duplicated_group_name(self, trans: ProvidesAppContext, group_name: str) -> None:
|
||||
if trans.sa_session.query(model.Group).filter(model.Group.table.c.name == group_name).first():
|
||||
raise Conflict(f"A group with name '{group_name}' already exists")
|
||||
|
||||
def _get_group(self, trans: ProvidesAppContext, encoded_id: EncodedDatabaseIdField) -> model.Group:
|
||||
decoded_group_id = self._decode_id(encoded_id)
|
||||
group = trans.sa_session.query(model.Group).get(decoded_group_id)
|
||||
if group is None:
|
||||
raise ObjectNotFound(f"Group with id {encoded_id} was not found.")
|
||||
return group
|
||||
|
||||
def _get_users_by_encoded_ids(self, trans: ProvidesAppContext, encoded_user_ids: List[EncodedDatabaseIdField]) -> List[model.User]:
|
||||
decoded_user_ids = self._decode_ids(encoded_user_ids)
|
||||
users = trans.sa_session.query(model.User).filter(model.User.table.c.id.in_(decoded_user_ids)).all()
|
||||
return users
|
||||
|
||||
def _get_roles_by_encoded_ids(self, trans: ProvidesAppContext, encoded_role_ids: List[EncodedDatabaseIdField]) -> List[model.Role]:
|
||||
decoded_role_ids = self._decode_ids(encoded_role_ids)
|
||||
roles = trans.sa_session.query(model.Role).filter(model.Role.table.c.id.in_(decoded_role_ids)).all()
|
||||
return roles
|
||||
@@ -18,8 +18,9 @@ from galaxy.managers import (
|
||||
datasets,
|
||||
secured,
|
||||
taggable,
|
||||
users
|
||||
users,
|
||||
)
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -40,12 +41,12 @@ class HDAManager(datasets.DatasetAssociationManager,
|
||||
# TODO: move what makes sense into DatasetManager
|
||||
# TODO: which of these are common with LDDAs and can be pushed down into DatasetAssociationManager?
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp, user_manager: users.UserManager):
|
||||
"""
|
||||
Set up and initialize other managers needed by hdas.
|
||||
"""
|
||||
super().__init__(app)
|
||||
self.user_manager = users.UserManager(app)
|
||||
self.user_manager = user_manager
|
||||
|
||||
def get_owned_ids(self, object_ids, history=None):
|
||||
"""Get owned IDs.
|
||||
@@ -247,7 +248,7 @@ class HDASerializer( # datasets._UnflattenedMetadataDatasetAssociationSerialize
|
||||
annotatable.AnnotatableSerializerMixin):
|
||||
model_manager_class = HDAManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.hda_manager = self.manager
|
||||
|
||||
@@ -506,7 +507,7 @@ class HDADeserializer(datasets.DatasetAssociationDeserializer,
|
||||
"""
|
||||
model_manager_class = HDAManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.hda_manager = self.manager
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ from galaxy.managers import (
|
||||
taggable
|
||||
)
|
||||
from galaxy.managers.collections_util import get_hda_and_element_identifiers
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util.zipstream import ZipstreamWrapper
|
||||
|
||||
|
||||
@@ -55,12 +56,6 @@ class HDCAManager(
|
||||
tag_assoc = model.HistoryDatasetCollectionTagAssociation
|
||||
annotation_assoc = model.HistoryDatasetCollectionAssociationAnnotationAssociation
|
||||
|
||||
def __init__(self, app):
|
||||
"""
|
||||
Set up and initialize other managers needed by hdcas.
|
||||
"""
|
||||
super().__init__(app)
|
||||
|
||||
def map_datasets(self, content, fn, *parents):
|
||||
"""
|
||||
Iterate over the datasets of a given collection, recursing into collections, and
|
||||
@@ -92,7 +87,7 @@ class DCESerializer(base.ModelSerializer):
|
||||
Serializer for DatasetCollectionElements.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.hda_serializer = hdas.HDASerializer(app)
|
||||
self.dc_serializer = DCSerializer(app, dce_serializer=self)
|
||||
@@ -126,7 +121,7 @@ class DCSerializer(base.ModelSerializer):
|
||||
Serializer for DatasetCollections.
|
||||
"""
|
||||
|
||||
def __init__(self, app, dce_serializer=None):
|
||||
def __init__(self, app: StructuredApp, dce_serializer=None):
|
||||
super().__init__(app)
|
||||
self.dce_serializer = dce_serializer or DCESerializer(app)
|
||||
|
||||
@@ -165,7 +160,7 @@ class DCASerializer(base.ModelSerializer):
|
||||
Base (abstract) Serializer class for HDCAs and LDCAs.
|
||||
"""
|
||||
|
||||
def __init__(self, app, dce_serializer=None):
|
||||
def __init__(self, app: StructuredApp, dce_serializer=None):
|
||||
super().__init__(app)
|
||||
self.dce_serializer = dce_serializer or DCESerializer(app)
|
||||
|
||||
@@ -220,7 +215,7 @@ class HDCASerializer(
|
||||
Serializer for HistoryDatasetCollectionAssociations.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.hdca_manager = HDCAManager(app)
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ from galaxy.managers import (
|
||||
history_contents,
|
||||
sharable
|
||||
)
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -37,11 +38,11 @@ class HistoryManager(sharable.SharableModelManager, deletable.PurgableManagerMix
|
||||
|
||||
# TODO: incorporate imp/exp (or alias to)
|
||||
|
||||
def __init__(self, app, *args, **kwargs):
|
||||
super().__init__(app, *args, **kwargs)
|
||||
self.hda_manager = hdas.HDAManager(app)
|
||||
self.contents_manager = history_contents.HistoryContentsManager(app)
|
||||
self.contents_filters = history_contents.HistoryContentsFilters(app)
|
||||
def __init__(self, app: StructuredApp, hda_manager: hdas.HDAManager, contents_manager: history_contents.HistoryContentsManager, contents_filters: history_contents.HistoryContentsFilters):
|
||||
super().__init__(app)
|
||||
self.hda_manager = hda_manager
|
||||
self.contents_manager = contents_manager
|
||||
self.contents_filters = contents_filters
|
||||
|
||||
def copy(self, history, user, **kwargs):
|
||||
"""
|
||||
@@ -171,7 +172,7 @@ class HistoryManager(sharable.SharableModelManager, deletable.PurgableManagerMix
|
||||
|
||||
class HistoryExportView:
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
|
||||
def get_exports(self, trans, history_id):
|
||||
@@ -223,13 +224,13 @@ class HistorySerializer(sharable.SharableModelSerializer, deletable.PurgableSeri
|
||||
model_manager_class = HistoryManager
|
||||
SINGLE_CHAR_ABBR = 'h'
|
||||
|
||||
def __init__(self, app, **kwargs):
|
||||
super().__init__(app, **kwargs)
|
||||
def __init__(self, app: StructuredApp, hda_manager: hdas.HDAManager, hda_serializer: hdas.HDASerializer, history_contents_serializer: history_contents.HistoryContentsSerializer):
|
||||
super().__init__(app)
|
||||
|
||||
self.history_manager = self.manager
|
||||
self.hda_manager = hdas.HDAManager(app)
|
||||
self.hda_serializer = hdas.HDASerializer(app)
|
||||
self.history_contents_serializer = history_contents.HistoryContentsSerializer(app)
|
||||
self.hda_manager = hda_manager
|
||||
self.hda_serializer = hda_serializer
|
||||
self.history_contents_serializer = history_contents_serializer
|
||||
|
||||
self.default_view = 'summary'
|
||||
self.add_view('summary', [
|
||||
@@ -442,7 +443,7 @@ class HistoryDeserializer(sharable.SharableModelDeserializer, deletable.Purgable
|
||||
"""
|
||||
model_manager_class = HistoryManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.history_manager = self.manager
|
||||
|
||||
|
||||
@@ -32,6 +32,7 @@ from galaxy.managers import (
|
||||
taggable,
|
||||
tools
|
||||
)
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -72,10 +73,10 @@ class HistoryContentsManager(containers.ContainerManagerMixin):
|
||||
)
|
||||
default_order_by = 'hid'
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
self.contained_manager = self.contained_class_manager_class(app)
|
||||
self.subcontainer_manager = self.subcontainer_class_manager_class(app)
|
||||
self.contained_manager = app[self.contained_class_manager_class]
|
||||
self.subcontainer_manager = app[self.subcontainer_class_manager_class]
|
||||
|
||||
# ---- interface
|
||||
def contained(self, container, filters=None, limit=None, offset=None, order_by=None, **kwargs):
|
||||
@@ -426,7 +427,7 @@ class HistoryContentsSerializer(base.ModelSerializer, deletable.PurgableSerializ
|
||||
"""
|
||||
model_manager_class = HistoryContentsManager
|
||||
|
||||
def __init__(self, app, **kwargs):
|
||||
def __init__(self, app: StructuredApp, **kwargs):
|
||||
super().__init__(app, **kwargs)
|
||||
|
||||
self.default_view = 'summary'
|
||||
|
||||
@@ -21,6 +21,7 @@ from galaxy.managers.collections import DatasetCollectionManager
|
||||
from galaxy.managers.datasets import DatasetManager
|
||||
from galaxy.managers.hdas import HDAManager
|
||||
from galaxy.managers.lddas import LDDAManager
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util import (
|
||||
defaultdict,
|
||||
ExecutionTimer,
|
||||
@@ -55,11 +56,11 @@ def get_path_key(path_tuple):
|
||||
|
||||
class JobManager:
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
self.dataset_manager = DatasetManager(app)
|
||||
|
||||
def job_lock(self):
|
||||
def job_lock(self) -> JobLock:
|
||||
return JobLock(active=self.app.job_manager.job_lock)
|
||||
|
||||
def update_job_lock(self, job_lock: JobLock):
|
||||
@@ -93,12 +94,18 @@ class JobManager:
|
||||
|
||||
class JobSearch:
|
||||
"""Search for jobs using tool inputs or other jobs"""
|
||||
def __init__(self, app):
|
||||
def __init__(
|
||||
self,
|
||||
app: StructuredApp,
|
||||
hda_manager: HDAManager,
|
||||
dataset_collection_manager: DatasetCollectionManager,
|
||||
ldda_manager: LDDAManager
|
||||
):
|
||||
self.app = app
|
||||
self.sa_session = app.model.context
|
||||
self.hda_manager = HDAManager(app)
|
||||
self.dataset_collection_manager = DatasetCollectionManager(app)
|
||||
self.ldda_manager = LDDAManager(app)
|
||||
self.hda_manager = hda_manager
|
||||
self.dataset_collection_manager = dataset_collection_manager
|
||||
self.ldda_manager = ldda_manager
|
||||
self.decode_id = self.app.security.decode_id
|
||||
|
||||
def by_tool_input(self, trans, tool_id, tool_version, param=None, param_dump=None, job_state='ok'):
|
||||
|
||||
@@ -3,6 +3,7 @@ import logging
|
||||
from galaxy import model, util
|
||||
from galaxy.managers import base as manager_base
|
||||
from galaxy.managers.datasets import DatasetAssociationManager
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -13,7 +14,7 @@ class LDDAManager(DatasetAssociationManager):
|
||||
"""
|
||||
model_class = model.LibraryDatasetDatasetAssociation
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
"""
|
||||
Set up and initialize other managers needed by lddas.
|
||||
"""
|
||||
|
||||
@@ -23,7 +23,6 @@ from galaxy.managers import (
|
||||
)
|
||||
from galaxy.managers.context import ProvidesAppContext
|
||||
from galaxy.schema.fields import EncodedDatabaseIdField
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util import (
|
||||
pretty_print_time_interval,
|
||||
unicodify,
|
||||
@@ -38,9 +37,6 @@ class LibraryManager:
|
||||
Interface/service object for interacting with libraries.
|
||||
"""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def get(self, trans, decoded_library_id, check_accessible=True):
|
||||
"""
|
||||
Get the library from the DB.
|
||||
@@ -329,10 +325,15 @@ class LibrariesManager:
|
||||
Interface/service object for sharing logic between controllers.
|
||||
"""
|
||||
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.folder_manager = folders.FolderManager()
|
||||
self.library_manager = LibraryManager()
|
||||
self.role_manager = roles.RoleManager(app)
|
||||
def __init__(
|
||||
self,
|
||||
folder_manager: folders.FolderManager,
|
||||
library_manager: LibraryManager,
|
||||
role_manager: roles.RoleManager,
|
||||
):
|
||||
self.folder_manager = folder_manager
|
||||
self.library_manager = library_manager
|
||||
self.role_manager = role_manager
|
||||
|
||||
def index(self, trans: ProvidesAppContext, deleted: Optional[bool] = False) -> List[Any]:
|
||||
"""Returns a list of summary data for all libraries.
|
||||
|
||||
@@ -11,6 +11,7 @@ from enum import Enum
|
||||
from html.entities import name2codepoint
|
||||
from html.parser import HTMLParser
|
||||
from typing import (
|
||||
Callable,
|
||||
List,
|
||||
Optional,
|
||||
)
|
||||
@@ -22,15 +23,15 @@ from pydantic import (
|
||||
|
||||
from galaxy import exceptions, model
|
||||
from galaxy.managers import base, sharable
|
||||
from galaxy.managers.hdas import HDAManager
|
||||
from galaxy.managers.context import ProvidesHistoryContext
|
||||
from galaxy.managers.markdown_util import (
|
||||
internal_galaxy_markdown_to_pdf,
|
||||
ready_galaxy_markdown_for_export,
|
||||
ready_galaxy_markdown_for_import,
|
||||
)
|
||||
from galaxy.managers.workflows import WorkflowsManager
|
||||
from galaxy.model.item_attrs import UsesAnnotations
|
||||
from galaxy.schema.fields import EncodedDatabaseIdField
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util import unicodify
|
||||
from galaxy.util.sanitize_html import sanitize_html
|
||||
|
||||
@@ -179,7 +180,7 @@ class PagesManager:
|
||||
Common interface/service logic for interactions with pages.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.manager = PageManager(app)
|
||||
|
||||
def index(self, trans, deleted: bool = False) -> PageSummaryList:
|
||||
@@ -282,11 +283,11 @@ class PageManager(sharable.SharableModelManager, UsesAnnotations):
|
||||
annotation_assoc = model.PageAnnotationAssociation
|
||||
rating_assoc = model.PageRatingAssociation
|
||||
|
||||
def __init__(self, app, *args, **kwargs):
|
||||
def __init__(self, app: StructuredApp):
|
||||
"""
|
||||
"""
|
||||
super().__init__(app, *args, **kwargs)
|
||||
self.workflow_manager = WorkflowsManager(app)
|
||||
super().__init__(app)
|
||||
self.workflow_manager = app.workflow_manager
|
||||
|
||||
def create(self, trans, payload):
|
||||
user = trans.get_user()
|
||||
@@ -379,7 +380,6 @@ class PageManager(sharable.SharableModelManager, UsesAnnotations):
|
||||
content = ready_galaxy_markdown_for_import(trans, content)
|
||||
else:
|
||||
raise exceptions.RequestParameterInvalidException("content_format [%s] must be either html or markdown" % content_format)
|
||||
|
||||
return content
|
||||
|
||||
def rewrite_content_for_export(self, trans, as_dict):
|
||||
@@ -406,7 +406,7 @@ class PageSerializer(sharable.SharableModelSerializer):
|
||||
model_manager_class = PageManager
|
||||
SINGLE_CHAR_ABBR = 'p'
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.page_manager = PageManager(app)
|
||||
|
||||
@@ -427,7 +427,7 @@ class PageDeserializer(sharable.SharableModelDeserializer):
|
||||
"""
|
||||
model_manager_class = PageManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.page_manager = self.manager
|
||||
|
||||
@@ -450,7 +450,7 @@ class PageContentProcessor(HTMLParser):
|
||||
'source', 'track', 'wbr'
|
||||
}
|
||||
|
||||
def __init__(self, trans, render_embed_html_fn):
|
||||
def __init__(self, trans, render_embed_html_fn: Callable):
|
||||
HTMLParser.__init__(self)
|
||||
self.trans = trans
|
||||
self.ignore_content = False
|
||||
@@ -627,28 +627,28 @@ PAGE_CLASS_MAPPING = {
|
||||
}
|
||||
|
||||
|
||||
def placeholderRenderForEdit(trans, item_class, item_id):
|
||||
def placeholderRenderForEdit(trans: ProvidesHistoryContext, item_class, item_id):
|
||||
return placeholderRenderForSave(trans, item_class, item_id, encode=True)
|
||||
|
||||
|
||||
def placeholderRenderForSave(trans, item_class, item_id, encode=False):
|
||||
def placeholderRenderForSave(trans: ProvidesHistoryContext, item_class, item_id, encode=False):
|
||||
encoded_item_id, decoded_item_id = get_page_identifiers(item_id, trans.app)
|
||||
item_name = ''
|
||||
if item_class == 'History':
|
||||
history = trans.sa_session.query(trans.model.History).get(decoded_item_id)
|
||||
history = trans.sa_session.query(model.History).get(decoded_item_id)
|
||||
history = base.security_check(trans, history, False, True)
|
||||
item_name = history.name
|
||||
elif item_class == 'HistoryDatasetAssociation':
|
||||
hda = trans.sa_session.query(trans.model.HistoryDatasetAssociation).get(decoded_item_id)
|
||||
hda_manager = HDAManager(trans.app)
|
||||
hda = trans.sa_session.query(model.HistoryDatasetAssociation).get(decoded_item_id)
|
||||
hda_manager = trans.app.hda_manager
|
||||
hda = hda_manager.get_accessible(decoded_item_id, trans.user)
|
||||
item_name = hda.name
|
||||
elif item_class == 'StoredWorkflow':
|
||||
wf = trans.sa_session.query(trans.model.StoredWorkflow).get(decoded_item_id)
|
||||
wf = trans.sa_session.query(model.StoredWorkflow).get(decoded_item_id)
|
||||
wf = base.security_check(trans, wf, False, True)
|
||||
item_name = wf.name
|
||||
elif item_class == 'Visualization':
|
||||
visualization = trans.sa_session.query(trans.model.Visualization).get(decoded_item_id)
|
||||
visualization = trans.sa_session.query(model.Visualization).get(decoded_item_id)
|
||||
visualization = base.security_check(trans, visualization, False, True)
|
||||
item_name = visualization.title
|
||||
class_shorthand = PAGE_CLASS_MAPPING[item_class]
|
||||
|
||||
@@ -53,9 +53,6 @@ class RoleManager(base.ModelManager):
|
||||
user_assoc = model.UserRoleAssociation
|
||||
group_assoc = model.GroupRoleAssociation
|
||||
|
||||
def __init__(self, app):
|
||||
super().__init__(app)
|
||||
|
||||
def get(self, trans: ProvidesUserContext, decoded_role_id):
|
||||
"""
|
||||
Method loads the role from the DB based on the given role id.
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import logging
|
||||
|
||||
from sqlalchemy import (
|
||||
and_,
|
||||
true,
|
||||
@@ -6,16 +8,23 @@ from sqlalchemy.orm import (
|
||||
joinedload,
|
||||
)
|
||||
|
||||
from galaxy.model.base import SharedModelMapping
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GalaxySessionManager:
|
||||
"""Manages GalaxySession."""
|
||||
|
||||
def __init__(self, model):
|
||||
def __init__(self, model: SharedModelMapping):
|
||||
self.model = model
|
||||
self.sa_session = model.context
|
||||
|
||||
def get_session_from_session_key(self, session_key: str):
|
||||
"""Returns GalaxySession if session_key is valid."""
|
||||
galaxy_session = self.model.session.query(self.model.GalaxySession).filter(
|
||||
# going through self.model since this can be used by Galaxy or Toolshed despite
|
||||
# type annotations
|
||||
galaxy_session = self.sa_session.query(self.model.GalaxySession).filter(
|
||||
and_(
|
||||
self.model.GalaxySession.table.c.session_key == session_key,
|
||||
self.model.GalaxySession.table.c.is_valid == true())
|
||||
|
||||
@@ -25,6 +25,7 @@ from galaxy.managers import (
|
||||
users
|
||||
)
|
||||
from galaxy.model import UserShareAssociation
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -40,7 +41,7 @@ class SharableModelManager(base.ModelManager, secured.OwnableManagerMixin, secur
|
||||
#: the single character abbreviation used in username_and_slug: e.g. 'h' for histories: u/user/h/slug
|
||||
SINGLE_CHAR_ABBR: Optional[str] = None
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
# user manager is needed to check access/ownership/admin
|
||||
self.user_manager = users.UserManager(app)
|
||||
|
||||
@@ -62,6 +62,7 @@ class TagsManager:
|
||||
"""
|
||||
tag_handler = GalaxyTagHandlerSession(trans.sa_session)
|
||||
id = trans.security.decode_id(payload.item_id)
|
||||
item_class = tag_handler.item_tag_assoc_info[payload.item_class].item_class
|
||||
item_class_name = str(payload.item_class)
|
||||
item_class = tag_handler.item_tag_assoc_info[item_class_name].item_class
|
||||
item = trans.sa_session.query(item_class).filter(item_class.id == id).first()
|
||||
return item
|
||||
|
||||
@@ -17,9 +17,6 @@ class DynamicToolManager(ModelManager):
|
||||
"""
|
||||
model_class = model.DynamicTool
|
||||
|
||||
def __init__(self, app):
|
||||
super().__init__(app)
|
||||
|
||||
def get_tool_by_uuid(self, uuid):
|
||||
dynamic_tool = self._one_or_none(
|
||||
self.query().filter(self.model_class.uuid == uuid)
|
||||
|
||||
@@ -30,6 +30,7 @@ from galaxy.security.validate_user_input import (
|
||||
validate_password,
|
||||
validate_publicname
|
||||
)
|
||||
from galaxy.structured_app import BasicApp, StructuredApp
|
||||
from galaxy.util.hash_util import new_secure_hash
|
||||
from galaxy.web import url_for
|
||||
|
||||
@@ -69,7 +70,7 @@ class UserManager(base.ModelManager, deletable.PurgableManagerMixin):
|
||||
# TODO: incorp BaseAPIController.validate_in_users_and_groups
|
||||
# TODO: incorp CreatesApiKeysMixin
|
||||
# TODO: incorporate UsesFormDefinitionsMixin?
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: BasicApp):
|
||||
self.model_class = app.model.User
|
||||
super().__init__(app)
|
||||
|
||||
@@ -240,14 +241,6 @@ class UserManager(base.ModelManager, deletable.PurgableManagerMixin):
|
||||
except exceptions.ObjectNotFound:
|
||||
return None
|
||||
|
||||
def by_email_like(self, email_with_wildcards, filters=None, order_by=None, **kwargs):
|
||||
"""
|
||||
Find a user searching with SQL wildcards.
|
||||
"""
|
||||
filters = self._munge_filters(self.model_class.email.like(email_with_wildcards), filters)
|
||||
order_by = order_by or (model.User.email, )
|
||||
return super().list(filters=filters, order_by=order_by, **kwargs)
|
||||
|
||||
def by_api_key(self, api_key, sa_session=None):
|
||||
"""
|
||||
Find a user by API key.
|
||||
@@ -353,6 +346,7 @@ class UserManager(base.ModelManager, deletable.PurgableManagerMixin):
|
||||
Create and return an API key for `user`.
|
||||
"""
|
||||
# TODO: seems like this should return the model
|
||||
# Also TODO: seems unused? drop and see what happens? -John
|
||||
return api_keys.ApiKeyManager(self.app).create_api_key(user)
|
||||
|
||||
def user_can_do_run_as(self, user) -> bool:
|
||||
@@ -617,7 +611,7 @@ class UserManager(base.ModelManager, deletable.PurgableManagerMixin):
|
||||
class UserSerializer(base.ModelSerializer, deletable.PurgableSerializerMixin):
|
||||
model_manager_class = UserManager
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
"""
|
||||
Convert a User and associated data to a dictionary representation.
|
||||
"""
|
||||
|
||||
@@ -8,6 +8,7 @@ import logging
|
||||
|
||||
from galaxy import model
|
||||
from galaxy.managers import sharable
|
||||
from galaxy.structured_app import StructuredApp
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -27,11 +28,6 @@ class VisualizationManager(sharable.SharableModelManager):
|
||||
annotation_assoc = model.VisualizationAnnotationAssociation
|
||||
rating_assoc = model.VisualizationRatingAssociation
|
||||
|
||||
def __init__(self, app, *args, **kwargs):
|
||||
"""
|
||||
"""
|
||||
super().__init__(app, *args, **kwargs)
|
||||
|
||||
# def copy( self, trans, visualization, user, **kwargs ):
|
||||
# """
|
||||
# """
|
||||
@@ -45,7 +41,7 @@ class VisualizationSerializer(sharable.SharableModelSerializer):
|
||||
model_manager_class = VisualizationManager
|
||||
SINGLE_CHAR_ABBR = 'v'
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
super().__init__(app)
|
||||
self.visualization_manager = self.manager
|
||||
|
||||
|
||||
@@ -26,6 +26,7 @@ from galaxy import (
|
||||
)
|
||||
from galaxy.jobs.actions.post import ActionBox
|
||||
from galaxy.model.item_attrs import UsesAnnotations
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.tools.parameters import (
|
||||
params_to_incoming,
|
||||
visit_input_values
|
||||
@@ -68,7 +69,7 @@ class WorkflowsManager:
|
||||
the galaxy.workflow module.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
|
||||
def get_stored_workflow(self, trans, workflow_id, by_stored_id=True):
|
||||
@@ -305,7 +306,7 @@ CreatedWorkflow = namedtuple("CreatedWorkflow", ["stored_workflow", "workflow",
|
||||
|
||||
class WorkflowContentsManager(UsesAnnotations):
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
self._resource_mapper_function = get_resource_mapper_function(app)
|
||||
|
||||
|
||||
@@ -370,6 +370,7 @@ class JobLike:
|
||||
class User(Dictifiable, RepresentById):
|
||||
use_pbkdf2 = True
|
||||
bootstrap_admin_user = False
|
||||
api_keys: 'List[APIKeys]'
|
||||
"""
|
||||
Data for a Galaxy user or admin and relations to their
|
||||
histories, credentials, and roles.
|
||||
|
||||
+47
-16
@@ -2,10 +2,13 @@
|
||||
Shared model and mapping code between Galaxy and Tool Shed, trying to
|
||||
generalize to generic database connections.
|
||||
"""
|
||||
import threading
|
||||
from contextvars import ContextVar
|
||||
from inspect import (
|
||||
getmembers,
|
||||
isclass
|
||||
)
|
||||
from typing import Dict, Type
|
||||
|
||||
from sqlalchemy import event
|
||||
from sqlalchemy.orm import (
|
||||
@@ -16,6 +19,14 @@ from sqlalchemy.orm import (
|
||||
from galaxy.util.bunch import Bunch
|
||||
|
||||
|
||||
# Create a ContextVar with mutable state, this allows sync tasks in the context
|
||||
# of a request (which run within a threadpool) to see changes to the ContextVar
|
||||
# state. See https://github.com/tiangolo/fastapi/issues/953#issuecomment-586006249
|
||||
# for details
|
||||
_request_state: Dict[str, str] = {}
|
||||
REQUEST_ID = ContextVar('request_id', default=_request_state.copy())
|
||||
|
||||
|
||||
# TODO: Refactor this to be a proper class, not a bunch.
|
||||
class ModelMapping(Bunch):
|
||||
|
||||
@@ -23,13 +34,13 @@ class ModelMapping(Bunch):
|
||||
self.engine = engine
|
||||
SessionLocal = sessionmaker(autoflush=False, autocommit=True)
|
||||
versioned_session(SessionLocal)
|
||||
context = scoped_session(SessionLocal)
|
||||
context = scoped_session(SessionLocal, scopefunc=self.request_scopefunc)
|
||||
# For backward compatibility with "context.current"
|
||||
# deprecated?
|
||||
context.current = context
|
||||
self._SessionLocal = SessionLocal
|
||||
self._session = context
|
||||
self.local_session = None
|
||||
self.session = context
|
||||
self.scoped_registry = context.registry
|
||||
|
||||
model_classes = {}
|
||||
for module in model_modules:
|
||||
@@ -42,22 +53,30 @@ class ModelMapping(Bunch):
|
||||
context.remove()
|
||||
context.configure(bind=engine)
|
||||
|
||||
def set_local_session(self):
|
||||
self.session = self._SessionLocal()
|
||||
def request_scopefunc(self):
|
||||
"""
|
||||
Return a value that is used as dictionary key for sqlalchemy's ScopedRegistry.
|
||||
|
||||
def dispose_local_session(self):
|
||||
self.session = None
|
||||
This ensures that threads or request contexts will receive a single identical session
|
||||
from the ScopedRegistry.
|
||||
"""
|
||||
return REQUEST_ID.get().get('request') or threading.get_ident()
|
||||
|
||||
@property
|
||||
def session(self):
|
||||
return self.local_session or self._session
|
||||
@staticmethod
|
||||
def set_request_id(request_id):
|
||||
# Set REQUEST_ID to a new dict.
|
||||
# This new ContextVar value will only be seen by the current asyncio context
|
||||
# and descendant threadpools, but not other threads or asyncio contexts.
|
||||
return REQUEST_ID.set({'request': request_id})
|
||||
|
||||
@session.setter
|
||||
def session(self, session):
|
||||
# For backward compatibility with "context.current"
|
||||
if session:
|
||||
session.current = session
|
||||
self.local_session = session
|
||||
def unset_request_id(self, request_id):
|
||||
# Unconditionally calling self.gx_app.model.session.remove()
|
||||
# would create a new session if the session was not accessed
|
||||
# in a request, so we check if there is a sqlalchemy session
|
||||
# for the current request in the registry.
|
||||
if request_id in self.scoped_registry.registry:
|
||||
self.scoped_registry.registry[request_id].close()
|
||||
del self.scoped_registry.registry[request_id]
|
||||
|
||||
@property
|
||||
def context(self):
|
||||
@@ -71,6 +90,18 @@ class ModelMapping(Bunch):
|
||||
return self.context
|
||||
|
||||
|
||||
class SharedModelMapping(ModelMapping):
|
||||
"""Model mapping containing references to classes shared between Galaxy and ToolShed.
|
||||
|
||||
Generally things can be more strongly typed when importing models directly, but we need
|
||||
a way to do app.model.<CLASS> for common code shared by the tool shed and Galaxy.
|
||||
"""
|
||||
User: Type
|
||||
GalaxySession: Type
|
||||
APIKeys: Type
|
||||
PasswordResetToken: Type
|
||||
|
||||
|
||||
def versioned_objects(iter):
|
||||
for obj in iter:
|
||||
if hasattr(obj, '__create_version__'):
|
||||
|
||||
@@ -25,7 +25,7 @@ class DatabaseHeartbeat:
|
||||
|
||||
@property
|
||||
def sa_session(self):
|
||||
return self.application_stack.app.model.context
|
||||
return self.application_stack.app.model.session
|
||||
|
||||
@property
|
||||
def server_name(self):
|
||||
|
||||
@@ -5,7 +5,7 @@ are encapsulated here.
|
||||
|
||||
import logging
|
||||
from threading import local
|
||||
from typing import Optional
|
||||
from typing import Optional, Type
|
||||
|
||||
from sqlalchemy import (
|
||||
and_,
|
||||
@@ -39,7 +39,7 @@ from sqlalchemy.sql import exists
|
||||
from sqlalchemy.types import BigInteger
|
||||
|
||||
from galaxy import model
|
||||
from galaxy.model.base import ModelMapping
|
||||
from galaxy.model.base import SharedModelMapping
|
||||
from galaxy.model.custom_types import (
|
||||
JSONType,
|
||||
MetadataType,
|
||||
@@ -2872,10 +2872,12 @@ def _workflow_invocation_update(self):
|
||||
model.WorkflowInvocation.update = _workflow_invocation_update # type: ignore
|
||||
|
||||
|
||||
class GalaxyModelMapping(ModelMapping):
|
||||
class GalaxyModelMapping(SharedModelMapping):
|
||||
security_agent: GalaxyRBACAgent
|
||||
thread_local_log: Optional[local]
|
||||
create_tables: bool
|
||||
User: Type
|
||||
GalaxySession: Type
|
||||
|
||||
|
||||
def init(file_path, url, engine_options=None, create_tables=False, map_install_models=False,
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import logging
|
||||
import re
|
||||
from typing import Dict
|
||||
|
||||
from sqlalchemy.orm.scoping import scoped_session
|
||||
from sqlalchemy.sql import select
|
||||
from sqlalchemy.sql.expression import func
|
||||
|
||||
@@ -26,7 +28,7 @@ class TagHandler:
|
||||
Manages CRUD operations related to tagging objects.
|
||||
"""
|
||||
|
||||
def __init__(self, sa_session):
|
||||
def __init__(self, sa_session: scoped_session) -> None:
|
||||
self.sa_session = sa_session
|
||||
# Minimum tag length.
|
||||
self.min_tag_len = 1
|
||||
@@ -39,7 +41,7 @@ class TagHandler:
|
||||
# Key-value separator.
|
||||
self.key_value_separators = "=:"
|
||||
# Initialize with known classes - add to this in subclasses.
|
||||
self.item_tag_assoc_info = {}
|
||||
self.item_tag_assoc_info: Dict[str, ItemTagAssocInfo] = {}
|
||||
|
||||
def create_tag_handler_session(self):
|
||||
# Creates a transient tag handler that avoids repeated flushes
|
||||
@@ -348,7 +350,7 @@ class TagHandler:
|
||||
|
||||
|
||||
class GalaxyTagHandler(TagHandler):
|
||||
def __init__(self, sa_session):
|
||||
def __init__(self, sa_session: scoped_session):
|
||||
from galaxy import model
|
||||
TagHandler.__init__(self, sa_session)
|
||||
self.item_tag_assoc_info["History"] = ItemTagAssocInfo(model.History,
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
"""Galaxy Quotas"""
|
||||
import abc
|
||||
import logging
|
||||
|
||||
import galaxy.util
|
||||
@@ -7,7 +6,7 @@ import galaxy.util
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class QuotaAgent(metaclass=abc.ABCMeta):
|
||||
class QuotaAgent(): # metaclass=abc.ABCMeta
|
||||
"""Abstraction around querying Galaxy for quota available and used.
|
||||
|
||||
Certain parts of the app that deal directly with modifying the quota assume more than
|
||||
@@ -21,7 +20,7 @@ class QuotaAgent(metaclass=abc.ABCMeta):
|
||||
the quota in other apps (LDAP maybe?) or via configuration files.
|
||||
"""
|
||||
|
||||
@abc.abstractmethod
|
||||
# TODO: make abstractmethod after they work better with mypy
|
||||
def get_quota(self, user):
|
||||
"""Return quota in bytes or None if no quota is set."""
|
||||
|
||||
@@ -34,7 +33,7 @@ class QuotaAgent(metaclass=abc.ABCMeta):
|
||||
quota_str = 'unlimited'
|
||||
return quota_str
|
||||
|
||||
@abc.abstractmethod
|
||||
# TODO: make abstractmethod after they work better with mypy
|
||||
def get_percent(self, trans=None, user=False, history=False, usage=False, quota=False):
|
||||
"""Return the percentage of any storage quota applicable to the user/transaction."""
|
||||
|
||||
@@ -228,7 +227,8 @@ class DatabaseQuotaAgent(QuotaAgent):
|
||||
return False
|
||||
|
||||
|
||||
def get_quota_agent(config, model):
|
||||
def get_quota_agent(config, model) -> QuotaAgent:
|
||||
quota_agent: QuotaAgent
|
||||
if config.enable_quotas:
|
||||
quota_agent = galaxy.quota.DatabaseQuotaAgent(model)
|
||||
else:
|
||||
|
||||
@@ -579,6 +579,10 @@ libraries:
|
||||
import_dir_btn:
|
||||
type: xpath
|
||||
selector: '//button[contains(text(), "Import")]'
|
||||
manage_dataset_permissions_btn: 'button[title="Permissions of ${name}"]'
|
||||
make_private_btn: '#make-private'
|
||||
access_dataset_roles: '.access_dataset_roles .multiselect__tag span'
|
||||
private_dataset_icon: '.fa-key'
|
||||
|
||||
labels:
|
||||
from_history: 'from History'
|
||||
|
||||
@@ -4,13 +4,16 @@ from typing import Any, Optional
|
||||
from kombu import Connection
|
||||
|
||||
from galaxy.auth import AuthManager
|
||||
from galaxy.datatypes.registry import Registry
|
||||
from galaxy.di import Container
|
||||
from galaxy.files import ConfiguredFileSources
|
||||
from galaxy.job_metrics import JobMetrics
|
||||
from galaxy.model.base import ModelMapping
|
||||
from galaxy.model.base import ModelMapping, SharedModelMapping
|
||||
from galaxy.model.mapping import GalaxyModelMapping
|
||||
from galaxy.model.security import GalaxyRBACAgent
|
||||
from galaxy.model.security import HostAgent
|
||||
from galaxy.model.tags import GalaxyTagHandler
|
||||
from galaxy.quota import QuotaAgent
|
||||
from galaxy.security.idencoding import IdEncodingHelper
|
||||
from galaxy.tool_util.deps.views import DependencyResolversView
|
||||
from galaxy.tool_util.verify import test_data
|
||||
@@ -20,7 +23,25 @@ from galaxy.webhooks import WebhooksRegistry
|
||||
from galaxy.workflow.trs_proxy import TrsProxy
|
||||
|
||||
|
||||
class StructuredApp:
|
||||
class BasicApp(Container):
|
||||
"""Stripped down version of the ``app`` shared between Galaxy and ToolShed.
|
||||
|
||||
Code that is shared between Galaxy and the Tool Shed should be annotated as
|
||||
using BasicApp instead of StructuredApp below.
|
||||
"""
|
||||
name: str
|
||||
config: Any
|
||||
application_stack: ApplicationStack
|
||||
model: SharedModelMapping
|
||||
security: IdEncodingHelper
|
||||
auth_manager: AuthManager
|
||||
toolbox: Any
|
||||
security_agent: Any
|
||||
quota_agent: QuotaAgent
|
||||
datatypes_registry: Registry
|
||||
|
||||
|
||||
class StructuredApp(BasicApp):
|
||||
"""Interface defining typed description of the Galaxy UniverseApplication.
|
||||
|
||||
Ideally nothing that depends on StructuredApp should require
|
||||
@@ -31,12 +52,9 @@ class StructuredApp:
|
||||
(cyclical imports), we're just setting the class attributes to
|
||||
Any.
|
||||
"""
|
||||
name: str
|
||||
is_webapp: bool # is_webapp will be set to true when building WSGI app
|
||||
new_installation: bool
|
||||
config: Any
|
||||
tag_handler: GalaxyTagHandler
|
||||
application_stack: ApplicationStack
|
||||
amqp_internal_connection_obj: Optional[Connection]
|
||||
dependency_resolvers_view: DependencyResolversView
|
||||
test_data_resolver: test_data.TestDataResolver
|
||||
@@ -45,10 +63,8 @@ class StructuredApp:
|
||||
job_metrics: JobMetrics
|
||||
model: GalaxyModelMapping
|
||||
install_model: ModelMapping
|
||||
security: IdEncodingHelper
|
||||
security_agent: GalaxyRBACAgent
|
||||
host_security_agent: HostAgent
|
||||
auth_manager: AuthManager
|
||||
trs_proxy: TrsProxy
|
||||
webhooks_registry: WebhooksRegistry
|
||||
|
||||
@@ -73,6 +89,10 @@ class StructuredApp:
|
||||
installed_repository_manager: Any # 'galaxy.tool_shed.galaxy_install.installed_repository_manager.InstalledRepositoryManager'
|
||||
workflow_scheduling_manager: Any # 'galaxy.workflow.scheduling_manager.WorkflowSchedulingManager'
|
||||
interactivetool_manager: Any
|
||||
job_manager: Any # galaxy.jobs.manager.JobManager
|
||||
user_manager: Any
|
||||
api_keys_manager: Any
|
||||
toolbox: Any
|
||||
|
||||
@property
|
||||
def is_job_handler(self) -> bool:
|
||||
pass
|
||||
|
||||
@@ -27,7 +27,6 @@ from galaxy import (
|
||||
)
|
||||
from galaxy.exceptions import ToolInputsNotReadyException
|
||||
from galaxy.job_execution import output_collect
|
||||
from galaxy.managers.jobs import JobSearch
|
||||
from galaxy.metadata import get_metadata_compute_strategy
|
||||
from galaxy.tool_shed.util.repository_util import get_installed_repository
|
||||
from galaxy.tool_shed.util.shed_util_common import set_image_paths
|
||||
@@ -553,7 +552,7 @@ class Tool(Dictifiable):
|
||||
# The job search is only relevant in a galaxy context, and breaks
|
||||
# loading tools into the toolshed for validation.
|
||||
if self.app.name == 'galaxy':
|
||||
self.job_search = JobSearch(app=self.app)
|
||||
self.job_search = self.app.job_search
|
||||
|
||||
def __getattr__(self, name):
|
||||
lazy_attributes = {
|
||||
|
||||
@@ -7,6 +7,7 @@ import tempfile
|
||||
import zlib
|
||||
from collections import defaultdict
|
||||
from threading import Lock
|
||||
from typing import Dict, List, Tuple
|
||||
|
||||
from sqlalchemy.orm import (
|
||||
defer,
|
||||
@@ -14,9 +15,13 @@ from sqlalchemy.orm import (
|
||||
)
|
||||
from sqlitedict import SqliteDict
|
||||
|
||||
from galaxy.model.tool_shed_install import ToolShedRepository
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.tools.toolbox.base import ToolConfRepository
|
||||
from galaxy.util import unicodify
|
||||
from galaxy.util.hash_util import md5_hash_file
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
CURRENT_TOOL_CACHE_VERSION = 0
|
||||
@@ -277,8 +282,11 @@ class ToolShedRepositoryCache:
|
||||
"""
|
||||
Cache installed ToolShedRepository objects.
|
||||
"""
|
||||
local_repositories: List[ToolConfRepository]
|
||||
repositories: List[ToolShedRepository]
|
||||
repos_by_tuple: Dict[Tuple[str, str, str], List[ToolConfRepository]]
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
# Contains ToolConfRepository objects created from shed_tool_conf.xml entries
|
||||
self.local_repositories = []
|
||||
@@ -294,10 +302,10 @@ class ToolShedRepositoryCache:
|
||||
def rebuild(self):
|
||||
try:
|
||||
session = self.app.install_model._SessionLocal()
|
||||
self.repositories = session.query(self.app.install_model.ToolShedRepository).options(
|
||||
defer(self.app.install_model.ToolShedRepository.metadata),
|
||||
self.repositories = session.query(ToolShedRepository).options(
|
||||
defer(ToolShedRepository.metadata),
|
||||
joinedload('tool_dependencies').subqueryload('tool_shed_repository').options(
|
||||
defer(self.app.install_model.ToolShedRepository.metadata)
|
||||
defer(ToolShedRepository.metadata)
|
||||
),
|
||||
).all()
|
||||
repos_by_tuple = defaultdict(list)
|
||||
|
||||
@@ -2,9 +2,10 @@ import errno
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
from typing import Dict
|
||||
|
||||
from galaxy import util
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.tools.data import TabularToolDataTable
|
||||
from galaxy.util.template import fill_template
|
||||
|
||||
@@ -16,7 +17,10 @@ DEFAULT_VALUE_TRANSLATION_TYPE = 'template'
|
||||
|
||||
|
||||
class DataManagers:
|
||||
def __init__(self, app, xml_filename=None):
|
||||
data_managers: Dict[str, 'DataManager']
|
||||
managed_data_tables: Dict[str, 'DataManager']
|
||||
|
||||
def __init__(self, app: StructuredApp, xml_filename=None):
|
||||
self.app = app
|
||||
self.data_managers = {}
|
||||
self.managed_data_tables = {}
|
||||
|
||||
@@ -40,6 +40,7 @@ class ValidationContext:
|
||||
# not appear to be an issue with the len_file_path config option.
|
||||
self.config.builds_file_path = None
|
||||
self.genome_builds = GenomeBuilds(self)
|
||||
self.job_search = None
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
@@ -32,6 +32,7 @@ from os.path import (
|
||||
relpath,
|
||||
sep as separator,
|
||||
)
|
||||
from pathlib import Path
|
||||
try:
|
||||
from pwd import getpwuid
|
||||
except ImportError:
|
||||
@@ -352,6 +353,9 @@ def external_chown(path, pwent, external_chown_script, description="file"):
|
||||
try:
|
||||
if not external_chown_script:
|
||||
raise ValueError('external_chown_script is not defined')
|
||||
if Path(path).owner() == pwent[0]:
|
||||
return True
|
||||
|
||||
cmd = shlex.split(external_chown_script)
|
||||
cmd.extend([path, pwent[0], str(pwent[3])])
|
||||
log.debug('Changing ownership of {} with: {}'.format(path, ' '.join(map(shlex.quote, cmd))))
|
||||
|
||||
@@ -3,6 +3,7 @@ import os
|
||||
import re
|
||||
import sys
|
||||
from json import loads
|
||||
from typing import Dict
|
||||
|
||||
from bx.seq.twobit import TwoBitFile
|
||||
|
||||
@@ -10,6 +11,7 @@ from galaxy.exceptions import (
|
||||
ObjectNotFound,
|
||||
ReferenceDataError,
|
||||
)
|
||||
from galaxy.structured_app import StructuredApp
|
||||
from galaxy.util.bunch import Bunch
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -195,10 +197,10 @@ class Genomes:
|
||||
Provides information about available genome data and methods for manipulating that data.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
def __init__(self, app: StructuredApp):
|
||||
self.app = app
|
||||
# Create list of genomes from app.genome_builds
|
||||
self.genomes = {}
|
||||
self.genomes: Dict[str, Genome] = {}
|
||||
# Store internal versions of data tables for twobit and __dbkey__
|
||||
self._table_versions = {'twobit': None, '__dbkeys__': None}
|
||||
self.reload_genomes()
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user