cwltool.process
===============

.. py:module:: cwltool.process

.. autoapi-nested-parse::

   Classes and methods relevant for all CWL Process types.



Attributes
----------

.. autoapisummary::

   cwltool.process.supportedProcessRequirements
   cwltool.process.cwl_files
   cwltool.process.salad_files
   cwltool.process.SCHEMA_CACHE
   cwltool.process.SCHEMA_FILE
   cwltool.process.SCHEMA_DIR
   cwltool.process.SCHEMA_ANY
   cwltool.process.custom_schemas
   cwltool.process.FILE_COUNT_WARNING
   cwltool.process.CWL_IANA


Classes
-------

.. autoapisummary::

   cwltool.process.LogAsDebugFilter
   cwltool.process.Process


Functions
---------

.. autoapisummary::

   cwltool.process.use_standard_schema
   cwltool.process.use_custom_schema
   cwltool.process.get_schema
   cwltool.process.shortname
   cwltool.process.stage_files
   cwltool.process.relocateOutputs
   cwltool.process.cleanIntermediate
   cwltool.process.add_sizes
   cwltool.process.fill_in_defaults
   cwltool.process.avroize_type
   cwltool.process.get_overrides
   cwltool.process.var_spool_cwl_detector
   cwltool.process.eval_resource
   cwltool.process.uniquename
   cwltool.process.nestdir
   cwltool.process.mergedirs
   cwltool.process.scandeps
   cwltool.process.compute_checksums


Module Contents
---------------

.. py:class:: LogAsDebugFilter(name, parent)

   Bases: :py:obj:`logging.Filter`

   .. autoapi-inheritance-diagram:: cwltool.process.LogAsDebugFilter
      :parts: 1


   Filter instances are used to perform arbitrary filtering of LogRecords.

   Loggers and Handlers can optionally use Filter instances to filter
   records as desired. The base filter class only allows events which are
   below a certain point in the logger hierarchy. For example, a filter
   initialized with "A.B" will allow events logged by loggers "A.B",
   "A.B.C", "A.B.C.D", "A.B.D" etc. but not "A.BB", "B.A.B" etc. If
   initialized with the empty string, all events are passed.


   .. py:attribute:: parent


   .. py:method:: filter(record)

      Determine if the specified record is to be logged.

      Returns True if the record should be logged, or False otherwise.
      If deemed appropriate, the record may be modified in-place.



.. py:data:: supportedProcessRequirements

.. py:data:: cwl_files
   :value: ('Base.yml', 'Workflow.yml', 'CommandLineTool.yml', 'CommonWorkflowLanguage.yml', 'Process.yml',...


.. py:data:: salad_files
   :value: ('metaschema.yml', 'metaschema_base.yml', 'salad.md', 'field_name.yml', 'import_include.md',...


.. py:data:: SCHEMA_CACHE
   :type:  dict[str, tuple[schema_salad.ref_resolver.Loader, schema_salad.avro.schema.Names | schema_salad.avro.schema.SchemaParseException, cwl_utils.types.CWLObjectType, schema_salad.ref_resolver.Loader]]

.. py:data:: SCHEMA_FILE
   :type:  cwl_utils.types.CWLObjectType | None
   :value: None


.. py:data:: SCHEMA_DIR
   :type:  cwl_utils.types.CWLObjectType | None
   :value: None


.. py:data:: SCHEMA_ANY
   :type:  cwl_utils.types.CWLObjectType | None
   :value: None


.. py:data:: custom_schemas
   :type:  dict[str, tuple[str, str]]

.. py:function:: use_standard_schema(version)

.. py:function:: use_custom_schema(version, name, text)

.. py:function:: get_schema(version)

.. py:function:: shortname(inputid)

.. py:function:: stage_files(pathmapper, stage_func = None, ignore_writable = False, symlink = True, secret_store = None, fix_conflicts = False)

   Link or copy files to their targets. Create them as needed.

   :raises WorkflowException: if there is a file staging conflict


.. py:function:: relocateOutputs(outputObj, destination_path, source_directories, action, fs_access, compute_checksum = True, path_mapper = PathMapper)

.. py:function:: cleanIntermediate(output_dirs)

.. py:function:: add_sizes(fsaccess, obj)

.. py:function:: fill_in_defaults(inputs, job, fsaccess)

   For each missing input in the input object, copy over the default.

   :raises WorkflowException: if a required input parameter is missing


.. py:function:: avroize_type(field_type, name_prefix = '')

   Add missing information to a type so that CWL types are valid.


.. py:function:: get_overrides(overrides, toolid)

   Combine overrides for the target tool ID.


.. py:function:: var_spool_cwl_detector(obj, item = None, obj_key = None)

   Detect any textual reference to /var/spool/cwl.


.. py:function:: eval_resource(builder, resource_req)

   Evaluate any CWL expressions inside a ResourceRequirement.


.. py:data:: FILE_COUNT_WARNING
   :value: 5000


.. py:class:: Process(toolpath_object, loadingContext)

   Bases: :py:obj:`cwltool.utils.HasReqsHints`

   .. autoapi-inheritance-diagram:: cwltool.process.Process
      :parts: 1


   Abstract CWL Process.


   .. py:attribute:: metadata
      :type:  cwl_utils.types.CWLObjectType


   .. py:attribute:: names


   .. py:attribute:: tool


   .. py:attribute:: requirements


   .. py:attribute:: hints


   .. py:attribute:: original_requirements


   .. py:attribute:: original_hints


   .. py:attribute:: doc_loader


   .. py:attribute:: doc_schema


   .. py:attribute:: formatgraph
      :type:  rdflib.Graph | None
      :value: None



   .. py:attribute:: schemaDefs
      :type:  collections.abc.MutableMapping[str, cwl_utils.types.CWLObjectType]


   .. py:attribute:: inputs_record_schema
      :type:  cwl_utils.types.CWLObjectType


   .. py:attribute:: outputs_record_schema
      :type:  cwl_utils.types.CWLObjectType


   .. py:attribute:: container_engine
      :value: 'docker'



   .. py:method:: evalResources(builder, runtimeContext)


   .. py:method:: checkRequirements(rec, supported_process_requirements)

      Check the presence of unsupported requirements.



   .. py:method:: validate_hints(avsc_names, hints, strict)

      Process the hints field.



   .. py:method:: visit(op)


   .. py:method:: job(job_order, output_callbacks, runtimeContext)
      :abstractmethod:



   .. py:method:: __str__()

      Return the id of this CWL process.



.. py:function:: uniquename(stem, names = None)

   Construct a thread-unique name using the given stem as a prefix.


.. py:function:: nestdir(base, deps)

   Add intermediate directory objects to preserve the relative layout.


.. py:function:: mergedirs(listing)

.. py:data:: CWL_IANA
   :value: 'https://www.iana.org/assignments/media-types/application/cwl'


.. py:function:: scandeps(base, doc, reffields, urlfields, loadref, urljoin = urllib.parse.urljoin, nestdirs = True)

   Search for external files references in a CWL document or input object.

   Looks for objects with 'class: File' or 'class: Directory' and
   adds them to the list of dependencies.

   :param base: the base URL for relative references.
   :param doc: a CWL document or input object
   :param urlfields: added as a File dependency
   :param reffields: field name like a workflow step 'run'; will be
     added as a dependency and also loaded (using the 'loadref'
     function) and recursively scanned for dependencies.  Those
     dependencies will be added as secondary files to the primary file.
   :param nestdirs: if true, create intermediate directory objects when
     a file is located in a subdirectory under the starting directory.
     This is so that if the dependencies are materialized, they will
     produce the same relative file system locations.
   :returns: A list of File or Directory dependencies


.. py:function:: compute_checksums(fs_access, fileobj)

