.. Quick Start Guide

=================
Quick Start Guide
=================

This Quick Start Guide serves as a fast, hands-on introduction to L7|ESP and will walk you through performing common L7|ESP activities. Specifically, this Quick Start Guide will teach you how to:

* Navigate L7|ESP's user-friendly user interface (UI).
* Create a simple Protocol that tracks values associated with Entities.
* Create a simple Pipeline that generates a Report after each Entity is processed.
* Create a Workflow that allows the Protocol and Pipeline to be used together to process Entities.
* Create a Project for managing Experiments.
* Register Entities and then use the registered Entities in an Experiment based on the Workflow.
* Update the Workflow to learn about versioning, data linking, and L7|ESP Expressions.

Login
------

Open the URL provided by your Admin in any web browser. Log in using the credentials provided by your Admin.

.. image:: images/log-in.png
   :height: 400px

.. hint::
   If you are a new user and this is your first time logging in, it is recommended that you immediately reset your original password for security purposes.

Create Standard Protocol
--------------------------

In this section you will create a Standard Protocol to collect 260:280 ratios for nucleic acid Entities. This Standard Protocol will later be used when creating a Workflow.

1. Click the **L7|Master** app. Click **Protocols** from the Processes menu to access the **Protocols** page.

  .. image:: images/protocols.png
      :width: 600px

2. Click **+ New Protocol**.

  .. image:: images/add-new-protocol.png

3. Enter "260:280 Quantification" in the **Name** field. Select **No** under the **Use in manufacturing** and select **Standard Protocol** as the **Protocol Type**.

  .. image:: images/name-select-protocol-type0123.png

4. Click **+ Add Field**.

5. Enter "Type" in the **Display Name** field. 

6. Select **Picklist** as the **Type**. Expand the Parameters section and enter "DNA" in the **Enter option 0** field. Enter "RNA" in the **Enter option 1** field.

 .. image:: images/enter-options-first-column.png	

7. Click **+ Add Field**. (Observe that the Type field is saved and present in the Fields view and a new Field Details panel opens).

  .. image:: images/add-column-897.png

8. Enter "260:280 Ratio" in the **Display Name** field. Select **Numbers only allowed** as the **Type**.

  .. image:: images/enter-options-second-column.png	

9. Click **Save**. 

You now have a simple spreadsheet with three fields for data entry: a field to select the nucleic acid Entity type (i.e., DNA or RNA), a field to enter the measured ratio, and a default Complete field.

.. image:: images/standard-protocol-example.png

.. _Create Task for Pipeline:

Create Task for Pipeline
--------------------------

In this section you will create a Task that will use a bash shell script to perform a purity test to determine if the measured ``ratio`` is within the accepted range for the Entity ``material`` (i.e., DNA or RNA). This Task will later be used when creating a Pipeline.

.. hint::
   It is recommended that programmers create Tasks for Pipelines.

1. From the **Analysis** app, click the **Tasks** tab. Click **+ New Task**. 

2. Enter "260-280 Purity Test" in the **Name** field. 

3. Copy the purity test code below (from line 1 to line 22). **Be aware the bc command line program referenced in this pipeline is not always available. If the command fails, try installing the bc program.**

  .. literalinclude:: purity.sh
        :language: bash
        :linenos:

  .. note::
      Observe that ``{{ material }}`` and ``{{ ratio }}`` have double curly braces around them. ``{{ ... }}`` denotes an L7|ESP Expression. L7|ESP Expressions are based on the Python programming language and can be any single-line Python statement (with some security restrictions).     
      Expressions are evaluated before a Task is executed, allowing you to supply parameters to Tasks (in this case, ``material`` and ``ratio``) and to perform simple computations. 
      
      For more information on expressions, refer to :ref:`L7|ESP-Expression-Language`. 

4. Paste the purity test code as shown below.
   
  .. image:: images/new-task-name.png

5. Click **Save**.

Create Registered File
------------------------

In this section you will create a Registered File, which will track the result of the purity test.

1. Register the ``result.html`` file generated by the Task by clicking **+ Add** below Output Files.

2. Enter "Result Report" in the **File Name** field. Enter ``html`` in the **File Type** field. Enter ``{{ 'result.html' }}`` in the **Expression** field. Click **Save**.

  .. image:: images/tasklibrary-registeroutputfile.png

3. Click **Save** to save the updated Task.

Every time the Task is executed, the ``result.html`` file generated in the Task directory is registered with L7|ESP and then added to the Task's provenance (and, by extension, the Pipeline's provenance and the Entity's provenance). To learn more about L7|ESP provenance system, refer to :ref:`Provenance System`.

The final 260:280 Purity Test Task looks like this:

.. image:: images/tasklibrary-final.png

.. note::
   Files are typically registered by **Tasks** when **Pipelines** are executed, but you can also register files using the command line tools. To register a file from the command line, use the ``l7`` command:
   
   .. code-block:: shell
   
    $ /path/to/l7 file register /path/to/file 

Create Pipeline
-------------------

In this section you will create a Pipeline, which will use the Task you created earlier, to generate quality control reports for DNA and RNA purification. This Pipeline will later be used to create a Pipeline Protocol.

1. From the **Analysis** app, click the **Pipelines** tab. Click **+ New Pipeline**.

  .. image:: images/new-pipeline.png

2. Enter "260-280 Purity Test" in the **Name** field. Click the blank Task box. 

  .. image:: images/taskbox.png
  
3. Start entering "260" and select "260-280 Purity Test" from the auto-complete list.

  .. image:: images/pipelinebuilder-taskbox.png

4. Click **Save**.


Test Pipeline
~~~~~~~~~~~~~~~~~~~

In this section you will test the new Pipeline you created.

1. Click **Run**.

  .. image:: images/pipelinebuilder-runn.png

2. Enter "DNA" for the ``material`` parameter. Enter "1.9" for the ``ratio`` parameter. Click **Run**.

  .. image:: images/parameters-run-17.png
        :width: 600 px

3. Click the **Monitor** tab. Click the "260:280 Purity Test" row for the DNA/1.9 run. The Pipeline Inspector on the right displays basic information about the executed Pipeline.

  .. image:: images/pipelinemonitor-inspector.png

  Navigate through the results as follows:

    * Click the vertical ellipsis and then click **View Details**. This will redirect you to a page that looks similar to the Pipeline Builder. The key difference is that the **Run Details** page displays actual details from the run. Click the Task (the "260-280 Purity Test" box) to view run-time details, including associated Task Files. The Task Inspector on the right has links to the complete **Task Script** (as executed on the server); **stderr** and **stdout** for the script; and also a link to the Registered File, **Result Report**.

    .. image:: images/view-run-details-INSERT.png
  
    * Click **Result Report...** to view the contents of the Registered File. You have now successfully tested the Pipeline.

    .. image:: images/result-report.png

Create Pipeline Report
---------------------------

The Pipeline and Registered File demonstrate how to create a simple Pipeline and make the results available in L7|ESP; however, locating and viewing Files generated by Pipelines through Run Details can be cumbersome, especially if the Pipeline generates many Files not used for Reports (e.g., such as those tracked for provenance purposes). Additionally, you may want to display data from multiple Registered Files at once. L7|ESP’s Report Builder provides such a service. 

In this section you will use the Registered File to create a Report for the Pipeline.

1. From the **Analysis** app, click Pipelines. Click **Edit** on the 260:280 Purity Test row.

  .. image:: images/pipeline-edit.png

2. Click the **Reports** tab to access the Report Builder.

3. From the **Report Elements** list, drag "HTML file" to the canvas area and drop it on  the **Drag and drop a new report element here** field.

  .. image:: images/drag-file-details.png

4. From the **Registered Files** list, drag "Result Report (Step 1: 260-280)" to the canvas area and drop it on the **Drag and drop a registered file here** field. 

  .. image:: images/drag-result-repor.png

5. Click **Save**.

  .. image:: images/rawfile-resultreport.png

Test Updated Pipeline
~~~~~~~~~~~~~~~~~~~~~~

In this section you will test the updated Pipeline in order to see the new Report.

1. From the **Analysis** app, click the Pipelines tab. Click **Run** on the 260:280 Purity Test row.

  .. image:: images/parameters-101.png

2. Enter "RNA" for the ``material`` parameter. Enter "1.5" for the ``ratio`` parameter. Click **Run**.

  .. image:: images/parameters-78.png
         :width: 600 px

3. Click the **Monitor** tab. Click the new "260:280 Purity Test" row for the RNA/1.5 run. Click **View** from the Pipeline Inspector on the right to view the RNA Report.  The Report shows a failed RNA Purity test.

  .. image:: images/pipelinereport-inspector-view.png

  .. important::
    From the Monitor tab, the first run (i.e., DNA/1.9) of the 260:280 Purity Test Pipeline does not have a Report associated with it, only the Registered Files. This absence is due to how L7|ESP *versions* Pipelines. When a Pipeline is run, the **Pipeline Instance** uses the current version of the Pipeline. If the Pipeline is updated later, only new Instances of the Pipeline will incorporate those updates. Refer to :ref:`Versioning System` for more information.
    

Create Pipeline Protocol
-------------------------

In this section you will create a Pipeline Protocol using the Pipeline you previously created in order to feed Entity data into the L7|ESP Pipeline system for computational analysis.

1. Open the **L7|Master** app. Click **Protocols** from the Processes menu to access the **Protocols** page.

  .. image:: images/protocols.png
      :width: 600px

2. Click **+ New Protocol**.

  .. image:: images/add-new-protocol.png

3. Enter "260:280 Report" in the **Display Name** field. Select **No** under **Use in manufacturing**. Select **Pipeline Protocol** as the **Protocol Type**. Select **260:280 Purity Test** as the **Pipeline**. 

  Observe that two new fields appear in the **Field View**: ``material`` and ``ratio``. These two new fields are for data entry, which will later be performed by filling out a Worksheet in the LIMS app. 

  .. image:: images/name-select-protocol-type.png

4. Click anywhere in the ``material`` field. The ``material`` field will be highlighted in blue. In the **Field Details** panel, select **Picklist** as the **Type**. Expand the Parameters section and enter "DNA" in the **Enter option 0** field. Enter "RNA" in the **Enter option 1** field.

  .. image:: images/material-column-details.png

5. Click anywhere in the ``ratio`` field. Now the ``ratio`` field will be highlighted in blue. In the **Field Details** panel, select **Numbers only allowed** as the **Type**.

  .. image:: images/edit-column-numbers-only.png

6. Click **Save**.

You have now created a Standard Protocol and Pipeline Protocol, which will be used to create a Workflow.

Create Workflow
-----------------

In this section you will create a simple quality control Workflow that determines the purity of a DNA or RNA Entity using the 260:280 spectrophotometric ratio. The image below shows that this Workflow will have two Protocols: a Standard Protocol to collect the 260:280 ratio and a Pipeline Protocol to generate a quality control Report.

.. image:: images/workflow-example-1.png
    :width: 400px

.. note::
   Workflow building is usually done by L7 Informatics during a customer's preliminary setup period; however, after following this section you will have the knowledge to create Workflows on your own, at any time. 

1. Open the **L7|Master** app. Click **Workflows** from the Processes menu.

  .. image:: images/workflows.png
       :width: 600px

2. Click **+ New Workflow**.

  .. image:: images/add-new-workflow.png

3. Enter "260:280 Purity Assay" in the **Display Name** field. Select **No** under **Use in manufacturing**.Click and drag the **260:280 Quantification** (from the **Protocol Library** panel) and drop in between the default Submitted and Completed boxes.

  .. image:: images/drag-drop-first-protocol.png

  .. note::
   The Submitted and Completed boxes are not Protocols or data entry fields - they are visual aids to display the status and quantity of an Entity Set as the individual Entities move through an Experiment. Specifically, the Submitted box displays the quantity of Entities submitted to a Workflow and the Completed box displays the quantity of Entities that have advanced through the entire Workflow.

4. Click and drag the **260:280 Report** (from the **Protocol Library** panel) and drop in between the 260:280 Quantification Protocol and the default Completed box.

  .. image:: images/drag-drop-second-protocol.png

5. Click **Save**.

Create Project
-------------------

In this section, you will create a Project. A Project is a collection of Experiments, grouped together for organizational purposes. You must create a Project before creating an Experiment. 

1. Open the **Projects** app. Click **+ New Project**.

2. Enter the Project name as "Quick Start."

3. Click **Save**.

  .. image:: images/quick-start-save.png
        :width: 400

.. _Experiment:

Create Experiment
-----------------

In this section you will create an Experiment, add the Experiment to the Project you previously created, and add/submit Entities to the Experiment.

1. On the row of the Quick Start Project you just created, click **+ Experiment**.

  .. image:: images/create-an-experiment-1.png
        :width: 600

2. Verify that **Workflow** is selected. Select **260:280 Purity Assay** from the **Workflow** drop-down. Enter "Hello Quick Start" as the **Name**. Click **Next**.

  .. image:: images/define-experiment.png
          :width: 600

3. Click **Add New Entities**. Select **Generic sample** from the **Entity Type** drop-down. Enter "4" in the **Number of Entities** field. Verify that **ESP SEQUENCE** is selected from the **Entity ID Sequence** drop-down. Click **Next**.

  .. image:: images/register-new-samples.png

  .. note::
      **Entity ID Sequence** is used to generate Entity names automatically as Entities are added.

4. Check the box to the left of **Entity ID** to select all 4 Entities. Click **Save and Submit**.

  .. image:: images/save-and-submit-samples.png

5. Click **Submit Entities** in the **Submit Confirmation** dialog.

The Experiment (with four Entities) has been submitted to the LIMS app.

.. _Worksheet:

Create Worksheet and Add Entities
----------------------------------

In this section you will create a Worksheet and add the Entities from the Experiment you previously created to the Worksheet. This Worksheet will later be used for data entry.

1. Open the **LIMS** app. Click the icon (shown below) that is located to the right of Hello Quick Start (4). This will open the **Add New Worksheet** dialog.

  .. image:: images/lims-newsheet-plusicon.png

2. Observe that the **Name** and **Workflow** fields are already populated. Update the **Name** to "Quick Start Sheet 1". Confirm that "260:280 Purity Assay" is selected as the **Workflow**. Click **Add Worksheet**.

  .. image:: images/newsheet-parameters.png
       :width: 400 px

3. To view the Worksheet, click the **Quick Start Sheet 1** link.

  .. image:: images/click-worksheet.png

  .. image:: images/lims-newworksheet-2.png

.. _process:

Process Entities and Perform Data Entry
-----------------------------------------

In this section you will practice processing Entities by performing data entry in the Worksheet you just created. 

1. In the **Type** column, select "DNA," "DNA," "RNA," and "RNA" from the drop-down list as shown below.
2. In the **260:280 Ratio** column, enter the values "1.7," "1.8," "1.9," and "2.0."
3. Select the "Complete" checkbox for each Entity.
4. Click **Save**.

  .. image:: images/filled-worksheet-1.png

  .. hint::
      Be sure to **Save** Worksheets as they are completed. L7|ESP stores the history of all Worksheet data, but clicking **Save** is necessary to ensure data goes into the system.

  .. important::
      To understand how the Entities are moving through the Experiment, open a new browser window and open the **Projects** app. Click the **Experiments** tab. Click the **Hello Quick Start** link under the Quick Start Project. 
      
      .. image:: images/samples-have-moved.png
      
      The blue 4 above 260:280 Report indicates that the 4 Entities you completed in the 260:280 Quantification Protocol have advanced to the 260:280 Report Protocol.
      
      .. image:: images/samples-have-moved-2.png
      
      The white 1 in a blue box indicates 1 Entity set. The red 0 indicates that 0 Entities failed the 260:280 Quantification Protocol. The green 4 indicates that 4 Entities passed the 260:280 Quantification Protocol.
      
      .. image:: images/samples-have-moved-3.png
      
      Return to the browser window with the LIMS app.

5. Click **Next** to enter information for the next Protocol.

  .. image:: images/complete-next-worksheet.png

6. In the **Material** column, select "DNA," "DNA," "RNA," and "RNA" from the drop-down list.
7. In the **Ratio** column, enter "1.7," "1.8," "1.9," and "2.0."
8. Click **Save**.

  .. image:: images/filled-worksheet-3.png

9. For the first Entity, click **Start Pipeline** to run the Pipeline that generates the Report. Click **Run** in the Run Pipeline verification dialog.

  .. image:: images/run-pipeline-3.png

10.  Entities can run simultaneously so click **Start Pipeline** and then **Run** in the Run Pipeline verification dialog for each of the remaining Entities. All **Complete** checkboxes will be checked and the **Start** column will say **Done** once the Pipelines are finished running.

  .. image:: images/run-pipeline-4.png

  .. important::
      To monitor Entities as they move through an Experiment, open the **Projects** app. Click **View Details** on the the Quick Start Project. Observe that the 4 Entities have advanced to the **Completed** stage. 
      
      .. image:: images/view-status.png
      
      Click on the Protocol box for the 260:280 Report. Observe that the Reports are directly accessible from the Inspector. The Reports are also accessible from the Entities page.

      .. image:: images/click-260-280-report-protocol.png

.. _Link Data Between Protocols:

Link Data between Protocols
-----------------------------

  .. note::
      For both Protocols in the Worksheet, you manually entered the same material type information and 260:280 Ratio values. Entering the same information more than one time creates an opportunity for data inconsistency - L7|ESP provides you with the tools to eliminate such a risk. Values can be passed between Protocols in a Workflow using **Data Links**. Through Data Linking, columns in a Protocol are populated with information/values from a Protocol that occurs beforehand in the same Workflow.

In this section you will update the Workflow to automatically populate the Pipeline Protocol with the information/values that get entered in the Standard Protocol.

1. Open the **L7|Master** app. Click **Workflows** from the Processes menu.

  .. image:: images/workflows.png
      :width: 600px

2. Click the **260:280 Purity Assay** Workflow.

  .. image:: images/create-data-link.png

3. Click the **260:280 Report** Protocol box. On the **material** row, click **+ Create** to create a Data Link to connect the values in this Protocol to the values entered in the 260:280 Quantification Protocol. 

  .. image:: images/create-data-link2.png

4. In the **Create Data Link** dialog, select **260:280 Quantification** as the Protocol. Select **Type** as the Column (the selected column is where the data will be linked). Click **Add Below** to generate an L7|ESP Expression based on the Protocol and Column selected. Click **Save**.

  .. image:: images/createdatalink-1.png
         :width: 400 px

  .. hint::
      Note that the :ref:`L7|ESP-Expression-Language` previously used to create parameters when setting up the Pipeline can also be used in other parts of L7|ESP. In this case, you can set the value of the ``material`` field to be an Expression that references a value in another Protocol.

5. On the **ratio** bar, click **Create**.

  .. image:: images/create-data-link4.png

6. In the **Create Data Link** dialog, select **260:280 Quantification** as the Protocol. Select **260:280 Ratio** as the Column. Click **Add Below**. Click **Save**.

  .. image:: images/createdatalink-44.png
         :width: 400 px

7. Observe that the Data Link now displays 2 Data Links. Click **Save** to save the updated Workflow.   

  .. image:: images/createdatalink-9.png

**By following this Quick Start Guide, you have successfully created an Entity-to-answer laboratory Workflow for the purity quality control test.**

Test Updated Workflow
---------------------------------------

To test your new L7|ESP knowledge and skills (as well as the Data Links), perform the steps below to test the updated Workflow.

1. Create a new Experiment_ under the Quick Start Project as follows: 

  - Select the 260:280 Purity Assay Workflow. 
  
  - Name the Experiment "Linked Quick Start." 
  
  - When adding Entities, choose the Use Existing Entities option and check 3 random Entities. 
  
  - Submit the 3 Entities.

2. Create a Worksheet_ for the Experiment named "Quick Start Sheet 2".

3. "Process_" the 3 Entities and perform data entry as follows:

  - Open the Quick Start Sheet 2 Worksheet. 

  - Complete the 260:280 Quantification Protocol by filling out the Type column and the 260:280 Ratio Column.

  - Complete the 3 Entities to advance them to the next Protocol (i.e., 260:280 Report Protocol). Observe that the material and ratio columns are prepopulated with the data from the 260:280 Quantification Protocol. This demonstrates that the data links are working.

  - Run the 3 Pipelines to complete the Experiment. 

  - Click Save. 
  
  - View the results of the Experiment.