Update tool

This commit is contained in:
Anup Kumar
2021-02-25 15:51:25 +01:00
parent 25b2ed272a
commit 37229bffe4
2 changed files with 96 additions and 82 deletions
@@ -1,9 +1,25 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "disabled-interest",
"metadata": {},
"source": [
"# Welcome to the Galaxy's GPU enabled Interactive Jupyter Lab for Machine Learning."
]
},
{
"cell_type": "markdown",
"id": "resident-livestock",
"metadata": {},
"source": [
"The jupyter notebook is powered by the latest Jupyter Lab (3.0.7) and Tensorflow (2.4.1) which can be used to prototype and develop machine learning and deep learning solutions executing on Galaxy's NVIDIA GPUs. The docker container used can be found at [website](https://github.com/anuprulez/ml-jupyter-notebook/tree/galaxy-inte-docker-ml-jupyter)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "several-election",
"execution_count": 26,
"id": "amber-tunisia",
"metadata": {},
"outputs": [],
"source": [
@@ -12,10 +28,18 @@
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"id": "lovely-adoption",
"metadata": {},
"source": [
"### Resources"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "rural-scoop",
"execution_count": 27,
"id": "central-roller",
"metadata": {},
"outputs": [
{
@@ -31,27 +55,25 @@
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "binary-blast",
"cell_type": "markdown",
"id": "available-metro",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Num CPUs Available: 1\n"
]
}
],
"source": [
"print(\"Num CPUs Available: \", len(tf.config.list_physical_devices('CPU')))"
"### An example neural network created with Tensorflow"
]
},
{
"cell_type": "markdown",
"id": "million-travel",
"metadata": {},
"source": [
"#### Collect data"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "wanted-exception",
"execution_count": 28,
"id": "nutritional-ghost",
"metadata": {},
"outputs": [],
"source": [
@@ -62,8 +84,8 @@
},
{
"cell_type": "code",
"execution_count": 5,
"id": "qualified-realtor",
"execution_count": 29,
"id": "transparent-equilibrium",
"metadata": {},
"outputs": [],
"source": [
@@ -73,8 +95,8 @@
},
{
"cell_type": "code",
"execution_count": 6,
"id": "considerable-private",
"execution_count": 30,
"id": "adaptive-hanging",
"metadata": {},
"outputs": [
{
@@ -100,8 +122,8 @@
},
{
"cell_type": "code",
"execution_count": 7,
"id": "extraordinary-description",
"execution_count": 31,
"id": "solved-samuel",
"metadata": {},
"outputs": [],
"source": [
@@ -112,8 +134,8 @@
},
{
"cell_type": "code",
"execution_count": 8,
"id": "controlling-paste",
"execution_count": 32,
"id": "embedded-charleston",
"metadata": {},
"outputs": [
{
@@ -139,10 +161,18 @@
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "answering-romantic",
"metadata": {},
"source": [
"#### Define neural network architecture"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "coordinate-jefferson",
"execution_count": 33,
"id": "enormous-guest",
"metadata": {},
"outputs": [],
"source": [
@@ -155,8 +185,8 @@
},
{
"cell_type": "code",
"execution_count": 10,
"id": "august-opinion",
"execution_count": 34,
"id": "architectural-arbor",
"metadata": {},
"outputs": [],
"source": [
@@ -168,7 +198,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "pending-madison",
"id": "nasty-difficulty",
"metadata": {},
"outputs": [
{
@@ -176,11 +206,23 @@
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1875/1875 [==============================] - 13s 6ms/step - loss: 0.6163 - accuracy: 0.7844\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.6184 - accuracy: 0.7847\n",
"Epoch 2/10\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.3804 - accuracy: 0.8625\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.3832 - accuracy: 0.8641\n",
"Epoch 3/10\n",
"1507/1875 [=======================>......] - ETA: 2s - loss: 0.3322 - accuracy: 0.8772"
"1875/1875 [==============================] - 10s 5ms/step - loss: 0.3398 - accuracy: 0.8766\n",
"Epoch 4/10\n",
"1875/1875 [==============================] - 10s 5ms/step - loss: 0.3143 - accuracy: 0.8850\n",
"Epoch 5/10\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.2972 - accuracy: 0.8895\n",
"Epoch 6/10\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.2777 - accuracy: 0.8959\n",
"Epoch 7/10\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.2676 - accuracy: 0.8991\n",
"Epoch 8/10\n",
"1875/1875 [==============================] - 11s 6ms/step - loss: 0.2572 - accuracy: 0.9029\n",
"Epoch 9/10\n",
"1418/1875 [=====================>........] - ETA: 2s - loss: 0.2515 - accuracy: 0.9051"
]
}
],
@@ -191,7 +233,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "personal-deployment",
"id": "designed-billy",
"metadata": {},
"outputs": [],
"source": [
@@ -203,7 +245,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "three-lighting",
"id": "random-backup",
"metadata": {},
"outputs": [],
"source": [
@@ -214,7 +256,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "plain-lease",
"id": "painful-catch",
"metadata": {},
"outputs": [],
"source": [
@@ -224,7 +266,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "specified-marshall",
"id": "accessory-approval",
"metadata": {},
"outputs": [],
"source": [
@@ -263,7 +305,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "prompt-toddler",
"id": "characteristic-vegetation",
"metadata": {},
"outputs": [],
"source": [
@@ -279,7 +321,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "naval-concentrate",
"id": "promising-mention",
"metadata": {},
"outputs": [],
"source": [
@@ -295,7 +337,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "amended-ethiopia",
"id": "hybrid-lover",
"metadata": {},
"outputs": [],
"source": [
@@ -317,7 +359,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "rental-attribute",
"id": "demanding-discipline",
"metadata": {},
"outputs": [],
"source": [
@@ -330,7 +372,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "original-architect",
"id": "killing-performance",
"metadata": {},
"outputs": [],
"source": [
@@ -343,7 +385,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "color-qatar",
"id": "located-picture",
"metadata": {},
"outputs": [],
"source": [
@@ -355,7 +397,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "pretty-channel",
"id": "insured-writing",
"metadata": {},
"outputs": [],
"source": [
@@ -366,7 +408,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "small-emission",
"id": "waiting-yahoo",
"metadata": {},
"outputs": [],
"source": [
@@ -1,9 +1,9 @@
<tool id="interactive_ml_tool_jupyter_notebook" tool_type="interactive" name="Interactive Jupyter Notebook for Machine learning" version="0.1">
<tool id="interactive_ml_tool_jupyter_notebook" tool_type="interactive" name="GPU enabled Interactive Jupyter Notebook for Machine Learning" version="0.1">
<requirements>
<container type="docker">docker.io/anupkumar/docker-ml-jupyterlab:galaxy-integration</container>
</requirements>
<entry_points>
<entry_point name="Jupyter Interactive Tool" requires_domain="True">
<entry_point name="GPU enabled Interactive Jupyter Notebook for Machine Learning" requires_domain="True">
<port>8888</port>
<url>ipython/lab</url>
</entry_point>
@@ -33,35 +33,7 @@
chown \$NB_USER ./default_tensorflow_notebook.ipynb &&
jupyter trust ./default_tensorflow_notebook.ipynb &&
/usr/local/bin/start-notebook.sh
##else:
##set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
##cp '$mode.ipynb' ./${cleaned_name}.ipynb &&
##jupyter trust ./${cleaned_name}.ipynb &&
##if $mode.run_it:
##jupyter nbconvert --to notebook --execute --output ./default_tensorflow_notebook.ipynb --allow-errors ./*.ipynb &&
##else:
##jupyter lab --allow-root --no-browser --NotebookApp.shutdown_button=True &&
###end if
##cp ./default_tensorflow_notebook.ipynb '$jupyter_notebook'
#end if
##if $mode.mode_select == 'scratch':
## copy default notebook
##cp '$__tool_directory__/default_notebook_ml.ipynb' ./ipython_galaxy_notebook_ml.ipynb &&
##jupyter trust ./ipython_galaxy_notebook_ml.ipynb &&
##jupyter lab --allow-root --no-browser --NotebookApp.shutdown_button=True &&
##cp ./ipython_galaxy_notebook_ml.ipynb '$jupyter_notebook'
##else:
##set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
##cp '$mode.ipynb' ./${cleaned_name}.ipynb &&
##jupyter trust ./${cleaned_name}.ipynb &&
##if $mode.run_it:
##jupyter nbconvert --to notebook --execute --output ./ipython_galaxy_notebook_ml.ipynb --allow-errors ./*.ipynb &&
##else:
##jupyter lab --allow-root --no-browser --NotebookApp.shutdown_button=True &&
##end if
##cp ./ipython_galaxy_notebook_ml.ipynb '$jupyter_notebook'
##end if
]]>
</command>
<inputs>
@@ -93,12 +65,12 @@
</test>
</tests>
<help>
The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations,
visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization,
machine learning, and much more.
JupyterLab is a next-generation web-based user interface for Project Jupyter. JupyterLab enables you to work with documents and activities such as Jupyter notebooks, text editors,
terminals, and custom components in a flexible, integrated, and extensible manner.
Galaxy offers you to use Jupyter Notebooks directly in Galaxy accessing and interacting with Galaxy datasets as you like. A very common use-case is to
do the heavy lifting and data reduction steps in Galaxy and the plotting and more `interactive` part on smaller datasets in Jupyter.
Galaxy offers you to use Jupyter Lab directly in Galaxy accessing and interacting with Galaxy datasets as you like. A very common use-case is to
do the heavy lifting such as performing computation on GPUs and data reduction steps in Galaxy and the plotting and more `interactive` part on smaller datasets in Jupyter Lab.
You can start with a new Jupyter notebook from scratch or load an already existing one, e.g. from your collegue and execute it on your dataset.
If you have a defined input dataset you can even execute a Jupyter notebook in a workflow, given that the notebook is writing the output back to the history.