Decision Support Pipeline Overview

Sara Juel

Updated May 11, 2022 12:37

With the Decision Support Pipeline (DSP), ILLiad can be configured to automate the borrowing request process for both articles and loans. Addons can run at different stages of the processes, allowing for data in the request to be updated, availability in different systems to be checked, and finally either automatically sending the request based on your own customizable business rules or leaving it for staff review. The Decision Support Pipeline feature consists of pipelines, stages, queues, addons, routing rules, and business rules. Yet setting up the feature only requires uploading the addons, enabling the pre-configured routing rules, and setting up the optional business rules. For step-by-step setup instructions, see Configuring the Decision Support Pipeline. Business rules are not required unless you are setting up automation and business logic based on target information (e.g., costs, local holdings, turnaround time, etc.) and the business rules are set in the Customization Manager.

DSP Hierarchy

The DSP feature has an entirely new workflow that utilizes new pipelines, stages, addons, queues, routing rules, and business rules.

Pipelines- The pipeline consists of stages and queues specific to each pipeline.
- Stages- The stages will kick-off the addons for information gathering, system availability, cost, and turnaround time gathering, and automation (if applicable).
  - Addons- The addons will explore and input information into the request form. If using business logic then the addons may also automatically send the request.
- Queues- There are 2 types of queues: queues that require staff review and queues that will automatically send or route the request to the next queue. The automation queues are simply there to record the process the request went through. The review queues require staff action such as Awaiting Copyright Clearance or error queues.
  - Business Rules- Custom rules used to compare different fulfillment options for each request.
  - Routing Rules- Routing rules will send requests into the pipeline and help move requests along the pipeline in the case the addon doesn't automatically send the request.

DSP Request Processing Order

The DSP will generally execute steps in the following order when processing requests, but may skip steps depending on your setup (for example, if no stage one addons are enabled on a pipeline, step 2 will be skipped):

A routing rule defined in the ILLiad Customization Manager's Routing table will send the request into one of the three DSP pipelines (Borrowing Article Pre-Copyright, Borrowing Article, or Borrowing Loan) by routing it to either the Awaiting Pre-Copyright Pipeline or Awaiting Request Pipeline queue

A request can also be manually routed into either queue to kickstart the DSP process.
The DSP will execute all stage 1 addons enabled for the specific pipeline that the request has entered and modify the ILLiad request details according to the information returned by each of these addons
The DSP will execute all stage 2 addons enabled for the specific pipeline that the request has entered and create FulfillmentData records for the request based on the information returned by each addon
The DSP will check the DSP business rules defined in the ILLiad Customization Manager for the relevant pipeline and run through the rules, in order, to check each rule's match string against the information stored in the ILLiad request's details and FulfillmentData records
As soon as one of the business rules is found to be a match for the request, the DSP will stop checking any subsequent rules for a match and apply that business rule's logic to the request as follows:
1. If the business rule's Rule Type is Automatic, the request will be routed to the Awaiting Pipeline Sending queue and sent for fulfillment to the specified Target addon if the rule's Target Type is System or Addon. If the rule's Target Type is Route, the request will be automatically routed to the specified Target queue.
2. If the business rule's Rule Type is Recommended, the request will be routed to either the Awaiting Pipeline Request Review or Awaiting Copyright Clearance - Pipeline queue (depending on if the pipeline is Post-Copyright or Pre-Copyright) where staff can review the recommended action generated from the business rule's logic before manually sending the request to the rule's specified Target
If no business rules match the request, the request will be routed to either the Awaiting Pipeline Request Review or Awaiting Copyright Clearance - Pipeline queue for staff to review the request for further processing

Tip: A business rule with the Target Type Route can be created and placed after all Target Type System or Addon business rules defined for a pipeline to catch and route requests that are unable to be fulfilled in the DSP to a non-DSP queue in ILLiad for further processing.

DSP Addons

The Addon form in ILLiad’s Customization Manager as of ILLiad 9.1 now allows users to upload addons with an Addon Type of “DecisionSupport”. This new addon type is similar to ILLiad’s Server addon type, except with two additional configurations (NVTGC and Pipeline).

Only the default addons listed below or addons that have been created specifically for the DSP feature will be supported. For additional information on creating addons for the DSP, see Decision Support Addon Development.

NVTGC: DecisionSupport addons are registered to a single NVTGC, meaning that the addon will only be able to see and process Requests within a specific NVTGC, allowing individual sites to control which addons they want to use in their pipeline.
Pipeline: DecisionSupport addons can be registered to a specific Decision Support Pipeline (e.g., BorrowingArticle), meaning that the addon will only be able to see and process Requests that are flowing through that particular pipeline.

Default Addons

Developed in conjunction with the 9.1 release, there are 3 default server addons configured specially for the DSP:

DOI Resolver - Stage 1 addon
Open Access Button - Stage 2 & 3 addon
Reprints Desk for Decision Support Pipeline - Stage 2 & 3 addon

Additional addons for the Decision Support Pipeline will be created periodically and added to the ILLiad Addon Directory.

Pipelines

When configuring DecisionSupport addons in Customization Manager, staff users will be responsible for selecting which pipelines should run the addon. There are currently 3 pipelines available in ILLiad’s Decision Support Pipeline. Each of these pipelines is associated with a designated set of Stages that will run the addons and add information or send the request, based on the Process Type (Borrowing, DocDel, Lending), Request Type (Article, Loan), and which queue it's in (input queue).

If any problem occurs during one of the stages, then the request will be sent to the appropriate error queue for staff review. Often times the request may just need to be kicked back into the input queue and run again.

Pre-Copyright

Borrowing Article Pre-Copyright- This pipeline applies to requests that are in the input queue of Awaiting Pre-Copyright Pipeline, have a Process Type of Borrowing, and a Request Type of Article.
- Stage(s): Information Gathering, Fulfillment Data Generation, Business Rule Evaluation.
- Error Queue: Pre-Copyright Pipeline Errors.

Post-Copyright

Borrowing Article: This pipeline applies to requests that are in the input queue of Awaiting Request Pipeline, have a Process Type of Borrowing, and a Request Type of Article.
- Stage(s): Information Gathering, Fulfillment Data Generation, Business Rule Evaluation
- Error Queue: Awaiting Pipeline Error Review
Borrowing Loan: This pipeline applies to requests that are in the input queue of Awaiting Request Pipeline, have a Process Type of Borrowing, and a Request Type of Loan.
- Stage(s): Information Gathering, Fulfillment Data Generation, Business Rule Evaluation
- Error Queue: Awaiting Pipeline Error Review

Stages

Decision Support addons are then executed through three stages for each pipeline, based on their pre-configured Pipelines and NVTGCs, and the pipeline stages that each addon has registered to.

Stage 1- Information Gathering: Information gathering addons can run and add additional information to your request form by filling in any blank fields. For example, an addon that looks up additional information for a request based on a DOI number. These addons can run in a round and will be able to wait on a prior addon to finish before executing.
Stage 2- Fulfillment Data Generation: System availability, cost, and turnaround time gathering. This is where the addons can check if it's available in your own or consortial holdings, via Open Access, OCLC, CCC, Open Access, Reprints Desk, and so on. If so, what are the costs and how fast will you get the loan/article? These addons will execute in parallel using the current information in the request to determine availability. This information is stored in a new data table in ILLiad that each addon can write their information to called FulfillmentData.
Stage 3- Automator: The built-in Automator runs a set of business rules which you define that rank your options based on cost, availability, turnaround time, and a match string against the request to determine where it should be sent. Then, based on your setup, it will either send the request for you or place it into Awaiting Pipeline Request Review (Post-Copyright)/Awaiting Copyright Clearance - Pipeline (Pre-Copyright), with a new display area that shows all of your options and which one it recommends.

Business Rules

The Business Rule Evaluation stage of the Decision Support Pipeline evaluates user-defined business rules against individual requests, allowing the System Manager to compare different fulfillment options for each request. The business rules work similarly to match strings for Routing Rules. Business rules can be added to the Customization Manager under Addons | DSP Business Rules. There are two rule types, Automatic and Recommended. The automatic rule type will automatically send the request if one of the external systems meet the criteria of the business rule. The recommended rule type will route the request to either the Awaiting Pipeline Request Review (Post-Copyright) or Awaiting Copyright Clearance - Pipeline (Pre-Copyright) queue for staff review. From there staff can see the information gathered from the addons and select the recommended action or an alternative action for sending the request. A business rule can also be created to route a request out of the Decision Support Pipeline to a non-DSP ILLiad queue.

For more information on business rules, including examples and use cases, see Business Rules Workflow and Examples.

FulfillmentData Table

After the addons run in stage 2, information pulled in from the external systems will be added to the new FulfillmentData table in the Request form and the Pipeline Request Review Form. This will allow staff or the business logic of the DSP feature to compare all variables before making decisions. The table will populate with the following information if provided by the external systems:

Source- this will usually be a system name like Rapid, Reprints Desk, etc.
Cost- the cost of requesting from this source for this item, in US dollars.
Turnaround Time- turnaround time for this item is available, in hours.
Format- a field for the item format that would be delivered.

Copyright Included- for articles if copyright is included or not required for this item.
Custom Data - a field that the addon can write information that can be acted on in the business rules, consisting of a set of key/value pairs in a JSON format.
Transaction Number (TN)- The ILLiad reference number that allows a user to identify and locate a request.
Availability- a yes or no response as to whether the item can be supplied by the source. The information is stored to help track an addons success or failure for statistical data.
Notes- any special notes the addon received from the source provider. For example, taxes not included in pricing.

New Queues

The following queues will need to be manually added to the CustomQueues table in the ILLiad Customization Manager located under System | CustonQueues before the Decision Support Pipeline can be used (these queues should have a ProcessType of Borrowing):

Awaiting Request Pipeline: This queue is used for loans as well as articles that have passed the copyright step.
Awaiting Pre-copyright Pipeline: This queue is used for articles that require a copyright clearance step.
Awaiting Copyright Clearance - Pipeline: This queue is used for articles that have passed through the Pre-copyright pipeline and still require copyright review. This queue opens automatically on the Copyright Clearance Form.
Awaiting Pipeline Request Review: This queue is where requests go when they are not automatically sent by the Automation Decision stage of the Decision Support Pipeline. This can either be because none of the business rules matched or because the business rule it matched was set to have a Recommend action.

The following queues may be used by the System Manager when processing requests through the DSP, but do not need to be added to the CustomQueues table:

Awaiting Pipeline Sending: This queue is where requests are placed for the system to automatically send them based on your business rules.
Awaiting Pipeline Error Review: This queue is where requests are placed when an issue is encountered while processing a Transaction in a post-copyright pipeline or during the Automation Decision stage of the Decision Support Pipeline.
Pre-Copyright Pipeline Errors: This queue is where requests that are in the pre-copyright pipeline are placed when an addon returns an error.

Routing Rules

You can configure ILLiad to send requests into the DSP automatically by creating new routing rules in the Customization Manager under System | Routing. The rules should be configured to pick up the request and move it into the input queue of the DSP pipeline based on the RequestType.

Manually routing requests to the Awaiting Pre-Copyright Pipeline queue for Articles or the Awaiting Request Pipeline queue for Loans will also jump-start the DSP feature.

Routing Rule Samples:

Route request to the Awaiting Pre-Copyright Pipeline queue in the Borrowing Article Pre-Copyright pipeline if RequestType= Article.
Route request to the Awaiting Request Pipeline queue in the Borrowing Loan pipeline if RequestType=Loan.

Once you have configured the Decision Support Pipeline feature, see Processing Support Requests in the Client for instructions on how to process a request with the new configuration.