Extract job-global resources requirements from input args.

  Args:
    args: parsed command-line arguments

  Returns:
    Resources object containing the requested resources for the job


Print status info as we wait for those jobs.

  Blocks until either all of the listed jobs succeed,
  or one of them fails.

  Args:
    provider: job service provider
    job_ids: a set of job IDs (string) to wait for
    poll_interval: integer seconds to wait between iterations
    stop_on_failure: whether to stop waiting if one of the tasks fails.

  Returns:
    Empty list if there was no error,
    a list of error messages from the failed tasks otherwise.


Wait for job and retry any tasks that fail.

  Stops retrying an individual task when: it succeeds, is canceled, or has been
  retried "retries" times.

  This function exits when there are no tasks running and there are no tasks
  eligible to be retried.

  Args:
    provider: job service provider
    job_id: a single job ID (string) to wait for
    poll_interval: integer seconds to wait between iterations
    retries: number of retries
    job_descriptor: job descriptor used to originally submit job

  Returns:
    Empty list if there was no error,
    a list containing an error message from a failed task otherwise.


A list with, for each job, its dominant task.

  The dominant task is the one that exemplifies its job's
  status. It is either:
  - the first (FAILURE or CANCELED) task, or if none
  - the first RUNNING task, or if none
  - the first SUCCESS task.

  Args:
    tasks: a list of tasks to consider

  Returns:
    A list with, for each job, its dominant task.


Waits until any of the listed jobs is not running.

  In particular, if any of the jobs sees one of its tasks fail,
  we count the whole job as failing (but do not terminate the remaining
  tasks ourselves).

  Args:
    provider: job service provider
    job_ids: a list of job IDs (string) to wait for
    poll_interval: integer seconds to wait between iterations

  Returns:
    A set of the jobIDs with still at least one running task.


Validates that job and task argument names do not overlap.


Helper function to return an appropriate set of mount parameters.


Convenience function simplifies construction of the logging uri.


Split a string into a pair, which can have one empty value.

  Args:
    pair_string: The string to be split.
    separator: The separator to be used for splitting.
    nullable_idx: The location to be set to null if the separator is not in the
                  input string. Should be either 0 or 1.

  Returns:
    A list containing the pair.

  Raises:
    IndexError: If nullable_idx is not 0 or 1.


Parses task parameters from a TSV.

  Args:
    tasks: Dict containing the path to a TSV file and task numbers to run
    variables, input, and output parameters as column headings. Subsequent
    lines specify parameter values, one row per job.
    retries: Number of retries allowed.
    input_file_param_util: Utility for producing InputFileParam objects.
    output_file_param_util: Utility for producing OutputFileParam objects.

  Returns:
    task_descriptors: an array of records, each containing the task-id,
    task-attempt, 'envs', 'inputs', 'outputs', 'labels' that defines the set of
    parameters for each task of the job.

  Raises:
    ValueError: If no job records were provided


Parse flags of key=value pairs and return a list of argclass.

  For pair variables, we need to:
     * split the input into name=value pairs (value optional)
     * Create the EnvParam object

  Args:
    labels: list of 'key' or 'key=value' strings.
    argclass: Container class for args, must instantiate with argclass(k, v).

  Returns:
    list of argclass objects.


Convert the timeout duration to seconds.

  The value must be of the form "<integer><unit>" where supported
  units are s, m, h, d, w (seconds, minutes, hours, days, weeks).

  Args:
    interval: A "<integer><unit>" string.
    valid_units: A list of supported units.

  Returns:
    A string of the form "<integer>s" or None if timeout is empty.


Produce a default variable name if none is specified.


Find the file provider for a URI.


Do basic validation of the uri, return the path and filename.


Return a valid docker_path from a Google Persistent Disk url.


Return a MountParam given a GCS bucket, disk image or local path.


Turn the specified name and value into a valid Google label.


For each task, ensure that each task param entry is not None.


Return a new dict with any empty items removed.

  Note that this is not a deep check. If d contains a dictionary which
  itself contains empty items, those are never checked.

  This method exists to make to_serializable() functions cleaner.
  We could revisit this some day, but for now, the serialized objects are
  stripped of empty values to keep the output YAML more compact.

  Args:
    d: a dictionary
    required: list of required keys (for example, TaskDescriptors always emit
      the "task-id", even if None)

  Returns:
    A dictionary with empty items removed.


Converts a task-id to the numeric task-id.

  Args:
    task_id: task-id in either task-n or n format

  Returns:
    n


Raise ValueError if the label is invalid.


Populate a JobDescriptor from the local provider's original meta.yaml.

    The local job provider had the first incarnation of a YAML file for each
    task. That idea was extended here in the JobDescriptor and the local
    provider adopted the JobDescriptor.to_yaml() call to write its meta.yaml.

    The JobDescriptor.from_yaml() detects if it receives a local provider's
    "v0" meta.yaml and calls this function.

    Args:
      job: an object produced from decoding meta.yaml.

    Returns:
      A JobDescriptor populated as best we can from the old meta.yaml.


Populate and return a JobDescriptor from a YAML string.


Returns the task_descriptor corresponding to task_id.


Return a dictionary of environment variables for the user container.


Returns a dict combining the field for job and task params.


Kill jobs or job tasks.

  This function separates ddel logic from flag parsing and user output. Users
  of ddel who intend to access the data programmatically should use this.

  Args:
    provider: an instantiated dsub provider.
    user_ids: a set of user ids who "own" the job(s) to delete.
    job_ids: a set of job ids to delete.
    task_ids: a set of task ids to delete.
    labels: a set of LabelParam, each must match the job(s) to be cancelled.
    create_time_min: a timezone-aware datetime value for the earliest create
                     time of a task, inclusive.
    create_time_max: a timezone-aware datetime value for the most recent create
                     time of a task, inclusive.

  Returns:
    list of job ids which were deleted.


Return the value for the specified action.


Return the environment for the operation.


Return the image for the operation.


Return all events of a particular type.


Generate formatted jobs individually, in order of create-time.

  Args:
    provider: an instantiated dsub provider.
    statuses: a set of status strings that eligible jobs may match.
    user_ids: a set of user strings that eligible jobs may match.
    job_ids: a set of job-id strings eligible jobs may match.
    job_names: a set of job-name strings eligible jobs may match.
    task_ids: a set of task-id strings eligible tasks may match.
    task_attempts: a set of task-attempt strings eligible tasks may match.
    labels: set of LabelParam that all tasks must match.
    create_time_min: a timezone-aware datetime value for the earliest create
                     time of a task, inclusive.
    create_time_max: a timezone-aware datetime value for the most recent create
                     time of a task, inclusive.
    max_tasks: (int) maximum number of tasks to return per dstat job lookup.
    page_size: the page size to use for each query to the backend. May be
               ignored by some provider implementations.
    summary_output: (bool) summarize the job list.

  Yields:
    Individual task dictionaries with associated metadata


Returns a list of zones based on any wildcard input.

  This function is intended to provide an easy method for producing a list
  of desired zones for a pipeline to run in.

  The Pipelines API default zone list is "any zone". The problem with
  "any zone" is that it can lead to incurring Cloud Storage egress charges
  if the GCE zone selected is in a different region than the GCS bucket.
  See https://cloud.google.com/storage/pricing#network-egress.

  A user with a multi-region US bucket would want to pipelines to run in
  a "us-*" zone.
  A user with a regional bucket in US would want to restrict pipelines to
  run in a zone in that region.

  Rarely does the specific zone matter for a pipeline.

  This function allows for a simple short-hand such as:
     [ "us-*" ]
     [ "us-central1-*" ]
  These examples will expand out to the full list of US and us-central1 zones
  respectively.

  Args:
    input_list: list of zone names/patterns

  Returns:
    A list of zones, with any wildcard zone specifications expanded.


Converts a datestamp from RFC3339 UTC to a datetime.

  Args:
    rfc3339_utc_string: a datetime string in RFC3339 UTC "Zulu" format

  Returns:
    A datetime.


Returns the job-id or job-id.task-id for the operation.


Cancel a batch of operations.

  Args:
    batch_fn: API-specific batch function.
    cancel_fn: API-specific cancel function.
    ops: A list of operations to cancel.

  Returns:
    A list of operations canceled and a list of error messages.


Specific check for auth error codes.

  Return True if we should retry.

  False otherwise.
  Args:
    exception: An exception to test for transience.

  Returns:
    True if we should retry. False otherwise.


Configures genomics API client.

  Args:
    api_name: Name of the Google API (for example: "genomics")
    api_version: Version of the API (for example: "v2alpha1")
    credentials: Credentials to be used for the gcloud API calls.

  Returns:
    A configured Google Genomics API client with appropriate credentials.


Executes operation.

    Args:
      api: The base API object

    Returns:
       A response body object


Returns a type from a snippit of python source. Should normally be
    something just like 'str' or 'Object'.

        arg_type            the source to be evaluated
        T                         the default type
        arg                     context of where this type was extracted
        sig                     context from where the arg was extracted

    Returns a type or a Type


Returns a jsonified response with the specified HTTP status code.

    The positional and keyword arguments are passed directly to the
    :func:`flask.jsonify` function which creates the response.


Performs the actual sending action and returns the result


Return the Exception data in a format for JSON-RPC


An `inspect.getargspec` with a relaxed sanity check to support Cython.

    Motivation:

        A Cython-compiled function is *not* an instance of Python's
        types.FunctionType.  That is the sanity check the standard Py2
        library uses in `inspect.getargspec()`.  So, an exception is raised
        when calling `argh.dispatch_command(cythonCompiledFunc)`.  However,
        the CyFunctions do have perfectly usable `.func_code` and
        `.func_defaults` which is all `inspect.getargspec` needs.

        This function just copies `inspect.getargspec()` from the standard
        library but relaxes the test to a more duck-typing one of having
        both `.func_code` and `.func_defaults` attributes.


Prompts user for input. Correctly handles prompt message encoding.


Encodes given value so it can be written to given file object.

    Value may be Unicode, binary string or any other data type.

    The exact behaviour depends on the Python version:

    Python 3.x

        `sys.stdout` is a `_io.TextIOWrapper` instance that accepts `str`
        (unicode) and breaks on `bytes`.

        It is OK to simply assume that everything is Unicode unless special
        handling is introduced in the client code.

        Thus, no additional processing is performed.

    Python 2.x

        `sys.stdout` is a file-like object that accepts `str` (bytes)
        and breaks when `unicode` is passed to `sys.stdout.write()`.

        We can expect both Unicode and bytes. They need to be encoded so as
        to match the file object encoding.

        The output is binary if the object doesn't explicitly require Unicode.


Adds types, actions, etc. to given argument specification.
    For example, ``default=3`` implies ``type=int``.

    :param arg: a :class:`argh.utils.Arg` instance


Declares an argument for given function. Does not register the function
    anywhere, nor does it modify the function in any way.

    The signature of the decorator matches that of
    :meth:`argparse.ArgumentParser.add_argument`, only some keywords are not
    required if they can be easily guessed (e.g. you don't have to specify type
    or action when an `int` or `bool` default value is supplied).

    Typical use cases:

    - In combination with :func:`expects_obj` (which is not recommended);
    - in combination with ordinary function signatures to add details that
      cannot be expressed with that syntax (e.g. help message).

    Usage::

        from argh import arg

        @arg('path', help='path to the file to load')
        @arg('--format', choices=['yaml','json'])
        @arg('-v', '--verbosity', choices=range(0,3), default=2)
        def load(path, something=None, format='json', dry_run=False, verbosity=1):
            loaders = {'json': json.load, 'yaml': yaml.load}
            loader = loaders[args.format]
            data = loader(args.path)
            if not args.dry_run:
                if verbosity < 1:
                    print('saving to the database')
                put_to_database(data)

    In this example:

    - `path` declaration is extended with `help`;
    - `format` declaration is extended with `choices`;
    - `dry_run` declaration is not duplicated;
    - `verbosity` is extended with `choices` and the default value is
      overridden.  (If both function signature and `@arg` define a default
      value for an argument, `@arg` wins.)

    .. note::

        It is recommended to avoid using this decorator unless there's no way
        to tune the argument's behaviour or presentation using ordinary
        function signatures.  Readability counts, don't repeat yourself.


Make a guess about the config file location an try loading it.


Validate a configuration key according to `section.item`.


Searches the given text for mentions and expands them.

    For example:
    "@source.nick" will be expanded to "@<source.nick source.url>".


Try loading given cache file.


Checks if specified URL is cached.


Retrieves tweets from the cache.


Tries to remove cached tweets.


Retrieve your personal timeline.


Get or set config item.


Return human-readable relative time string.


Copy the Query object, optionally replacing the filters, order_by, or
        limit information on the copy. This is mostly an internal detail that
        you can ignore.


Returns only the first result from the query, if any.


This function handles all on_delete semantics defined on OneToMany columns.

    This function only exists because 'cascade' is *very* hard to get right.


Performs the actual prefix, suffix, and pattern match operations.


Estimates the total work necessary to calculate the prefix match over the
    given index with the provided prefix.


Search for model ids that match the provided filters.

        Arguments:

            * *filters* - A list of filters that apply to the search of one of
              the following two forms:

                1. ``'column:string'`` - a plain string will match a word in a
                   text search on the column

                .. note:: Read the documentation about the ``Query`` object
                  for what is actually passed during text search

                2. ``('column', min, max)`` - a numeric column range search,
                   between min and max (inclusive by default)

                .. note:: Read the documentation about the ``Query`` object
                  for information about open-ended ranges

                3. ``['column:string1', 'column:string2']`` - will match any
                   of the provided words in a text search on the column

                4. ``Prefix('column', 'prefix')`` - will match prefixes of
                   words in a text search on the column

                5. ``Suffix('column', 'suffix')`` - will match suffixes of
                   words in a text search on the column

                6. ``Pattern('column', 'pattern')`` - will match patterns over
                   words in a text search on the column

            * *order_by* - A string that names the numeric column by which to
              sort the results by. Prefixing with '-' will return results in
              descending order

            .. note:: While you can technically pass a non-numeric index as an
              *order_by* clause, the results will basically be to order the
              results by string comparison of the ids (10 will come before 2).

            .. note:: If you omit the ``order_by`` argument, results will be
              ordered by the last filter. If the last filter was a text
              filter, see the previous note. If the last filter was numeric,
              then results will be ordered by that result.

            * *offset* - A numeric starting offset for results
            * *count* - The maximum number of results to return from the query


This utility function will iterate over all entities of a provided model,
    refreshing their indices. This is primarily useful after adding an index
    on a column.

    Arguments:

        * *model* - the model whose entities you want to reindex
        * *block_size* - the maximum number of entities you want to fetch from
          Redis at a time, defaulting to 100

    This function will yield its progression through re-indexing all of your
    entities.

    Example use::

        for progress, total in refresh_indices(MyModel, block_size=200):
            print "%s of %s"%(progress, total)

    .. note:: This uses the session object to handle index refresh via calls to
      ``.commit()``. If you have any outstanding entities known in the
      session, they will be committed.


This utility function will clean out old index data that was accidentally
    left during item deletion in rom versions <= 0.27.0 . You should run this
    after you have upgraded all of your clients to version 0.28.0 or later.

    Arguments:

        * *model* - the model whose entities you want to reindex
        * *block_size* - the maximum number of items to check at a time
          defaulting to 100

    This function will yield its progression through re-checking all of the
    data that could be left over.

    Example use::

        for progress, total in clean_old_index(MyModel, block_size=200):
            print "%s of %s"%(progress, total)


Adds an entity to the session.


... Actually write data to Redis. This is an internal detail. Please don't
    call me directly.


Deletes the entity immediately. Also performs any on_delete operations
        specified as part of column definitions.


Will fetch one or more entities of this type from the session or
        Redis.

        Used like::

            MyModel.get(5)
            MyModel.get([1, 6, 2, 4])

        Passing a list or a tuple will return multiple entities, in the same
        order that the ids were passed.


Parse the options, set defaults and then fire up PhantomJS.


Call PhantomJS with the specified flags and options.


Release, incrementing the internal counter by one.


Register an approximation of memory used by FTP server process
    and all of its children.


Connect to FTP server, login and return an ftplib.FTP instance.


Decorator. Bring coroutine result up, so it can be used as async context

    ::

        >>> async def foo():
        ...
        ...     ...
        ...     return AsyncContextInstance(...)
        ...
        ... ctx = await foo()
        ... async with ctx:
        ...
        ...     # do

    ::

        >>> @async_enterable
        ... async def foo():
        ...
        ...     ...
        ...     return AsyncContextInstance(...)
        ...
        ... async with foo() as ctx:
        ...
        ...     # do
        ...
        ... ctx = await foo()
        ... async with ctx:
        ...
        ...     # do


Context manager with threading lock for set locale on enter, and set it
    back to original state on exit.

    ::

        >>> with setlocale("C"):
        ...     ...


Count `data` for throttle

        :param data: bytes of data for count
        :type data: :py:class:`bytes`

        :param start: start of read/write time from
            :py:meth:`asyncio.BaseEventLoop.time`
        :type start: :py:class:`float`


Set throttle limit

        :param value: bytes per second
        :type value: :py:class:`int` or :py:class:`None`


Parsing directory server response.

        :param s: response line
        :type s: :py:class:`str`

        :rtype: :py:class:`pathlib.PurePosixPath`


Parsing Microsoft Windows `dir` output

        :param b: response line
        :type b: :py:class:`bytes` or :py:class:`str`

        :return: (path, info)
        :rtype: (:py:class:`pathlib.PurePosixPath`, :py:class:`dict`)


Create stream for write data to `destination` file.

        :param destination: destination path of file on server side
        :type destination: :py:class:`str` or :py:class:`pathlib.PurePosixPath`

        :param offset: byte offset for stream start position
        :type offset: :py:class:`int`

        :rtype: :py:class:`aioftp.DataConnectionThrottleStreamIO`


Compute jenks natural breaks on a sequence of `values`, given `nb_class`,
    the number of desired class.

    Parameters
    ----------
    values : array-like
        The Iterable sequence of numbers (integer/float) to be used.
    nb_class : int
        The desired number of class (as some other functions requests
        a `k` value, `nb_class` is like `k` + 1). Have to be lesser than
        the length of `values` and greater than 2.

    Returns
    -------
    breaks : tuple of floats
        The computed break values, including minimum and maximum, in order
        to have all the bounds for building `nb_class` class,
        so the returned tuple has a length of `nb_class` + 1.

    Examples
    --------
    Using nb_class = 3, expecting 4 break values , including min and max :

    >>> jenks_breaks(
            [1.3, 7.1, 7.3, 2.3, 3.9, 4.1, 7.8, 1.2, 4.3, 7.3, 5.0, 4.3],
            nb_class = 3)  # Should output (1.2, 2.3, 5.0, 7.8)


Copy the contents of the screen to PIL image memory.

    :param bbox: optional bounding box (x1,y1,x2,y2)
    :param childprocess: pyscreenshot can cause an error,
            if it is used on more different virtual displays
            and back-end is not in different process.
            Some back-ends are always different processes: scrot, imagemagick
            The default is False if the program was started inside IDLE,
            otherwise it is True.
    :param backend: back-end can be forced if set (examples:scrot, wx,..),
                    otherwise back-end is automatic


Open a Mapchete process.

    Parameters
    ----------
    config : MapcheteConfig object, config dict or path to mapchete file
        Mapchete process configuration
    mode : string
        * ``memory``: Generate process output on demand without reading
          pre-existing data or writing new data.
        * ``readonly``: Just read data without processing new data.
        * ``continue``: (default) Don't overwrite existing output.
        * ``overwrite``: Overwrite existing output.
    zoom : list or integer
        process zoom level or a pair of minimum and maximum zoom level
    bounds : tuple
        left, bottom, right, top process boundaries in output pyramid
    single_input_file : string
        single input file if supported by process
    with_cache : bool
        process output data cached in memory

    Returns
    -------
    Mapchete
        a Mapchete process object


Determine zoom levels.


Worker function running the process.


Yield process tiles.

        Tiles intersecting with the input data bounding boxes as well as
        process bounds, if provided, are considered process tiles. This is to
        avoid iterating through empty tiles.

        Parameters
        ----------
        zoom : integer
            zoom level process tiles should be returned from; if none is given,
            return all process tiles

        yields
        ------
        BufferedTile objects


Process a large batch of tiles.

        Parameters
        ----------
        process : MapcheteProcess
            process to be run
        zoom : list or int
            either single zoom level or list of minimum and maximum zoom level;
            None processes all (default: None)
        tile : tuple
            zoom, row and column of tile to be processed (cannot be used with
            zoom)
        multi : int
            number of workers (default: number of CPU cores)
        max_chunksize : int
            maximum number of process tiles to be queued for each worker;
            (default: 1)


Process a large batch of tiles and yield report messages per tile.

        Parameters
        ----------
        zoom : list or int
            either single zoom level or list of minimum and maximum zoom level;
            None processes all (default: None)
        tile : tuple
            zoom, row and column of tile to be processed (cannot be used with
            zoom)
        multi : int
            number of workers (default: number of CPU cores)
        max_chunksize : int
            maximum number of process tiles to be queued for each worker;
            (default: 1)


Run the Mapchete process.

        Execute, write and return data.

        Parameters
        ----------
        process_tile : Tile or tile index tuple
            Member of the process tile pyramid (not necessarily the output
            pyramid, if output has a different metatiling setting)

        Returns
        -------
        data : NumPy array or features
            process output


Extract data from tile.


Calculate hillshading from elevation data.

        Parameters
        ----------
        elevation : array
            input elevation data
        azimuth : float
            horizontal angle of light source (315: North-West)
        altitude : float
            vertical angle of light source (90 would result in slope shading)
        z : float
            vertical exaggeration factor
        scale : float
            scale factor of pixel size units versus height units (insert 112000
            when having elevation values in meters in a geodetic projection)

        Returns
        -------
        hillshade : array


Extract contour lines from elevation data.

        Parameters
        ----------
        elevation : array
            input elevation data
        interval : integer
            elevation value interval when drawing contour lines
        field : string
            output field name containing elevation value
        base : integer
            elevation base value the intervals are computed from

        Returns
        -------
        contours : iterable
            contours as GeoJSON-like pairs of properties and geometry


Clip array by geometry.

        Parameters
        ----------
        array : array
            raster data to be clipped
        geometries : iterable
            geometries used to clip source array
        inverted : bool
            invert clipping (default: False)
        clip_buffer : int
            buffer (in pixels) geometries before applying clip

        Returns
        -------
        clipped array : array


Create tile pyramid out of input raster.


Create a tile pyramid out of an input raster dataset.


Determine minimum and maximum zoomlevel.


Validate whether value is found in config and has the right type.

    Parameters
    ----------
    config : dict
        configuration dictionary
    values : list
        list of (str, type) tuples of values and value types expected in config

    Returns
    -------
    True if config is valid.

    Raises
    ------
    Exception if value is not found or has the wrong type.


Return hash of x.


Validate and return zoom levels.


Snaps bounds to tiles boundaries of specific zoom level.

    Parameters
    ----------
    bounds : bounds to be snapped
    pyramid : TilePyramid
    zoom : int

    Returns
    -------
    Bounds(left, bottom, right, top)


Clips bounds by clip.

    Parameters
    ----------
    bounds : bounds to be clipped
    clip : clip bounds

    Returns
    -------
    Bounds(left, bottom, right, top)


Return parameter dictionary per zoom level.


Return the element filtered by zoom level.

        - An input integer or float gets returned as is.
        - An input string is checked whether it starts with "zoom". Then, the
          provided zoom level gets parsed and compared with the actual zoom
          level. If zoom levels match, the element gets returned.
        TODOs/gotchas:
        - Elements are unordered, which can lead to unexpected results when
          defining the YAML config.
        - Provided zoom levels for one element in config file are not allowed
          to "overlap", i.e. there is not yet a decision mechanism implemented
          which handles this case.


Return element only if zoom condition matches with config string.


Flatten dict tree into dictionary where keys are paths of old dict.


Reverse tree flattening.


Process bounds this process is currently initialized with.

        This gets triggered by using the ``init_bounds`` kwarg. If not set, it will
        be equal to self.bounds.


Effective process bounds required to initialize inputs.

        Process bounds sometimes have to be larger, because all intersecting process
        tiles have to be covered as well.


Output object of driver.


Input items used for process stored in a dictionary.

        Keys are the hashes of the input parameters, values the respective
        InputData classes.


Optional baselevels configuration.

        baselevels:
            min: <zoom>
            max: <zoom>
            lower: <resampling method>
            higher: <resampling method>


Return configuration parameters snapshot for zoom as dictionary.

        Parameters
        ----------
        zoom : int
            zoom level

        Returns
        -------
        configuration snapshot : dictionary
        zoom level dependent process configuration


Return process bounding box for zoom level.

        Parameters
        ----------
        zoom : int or None
            if None, the union of all zoom level areas is returned

        Returns
        -------
        process area : shapely geometry


Generate indexes for given zoom level.

    Parameters
    ----------
    mp : Mapchete object
        process output to be indexed
    out_dir : path
        optionally override process output directory
    zoom : int
        zoom level to be processed
    geojson : bool
        generate GeoJSON index (default: False)
    gpkg : bool
        generate GeoPackage index (default: False)
    shapefile : bool
        generate Shapefile index (default: False)
    txt : bool
        generate tile path list textfile (default: False)
    vrt : bool
        GDAL-style VRT file (default: False)
    fieldname : str
        field name which contains paths of tiles (default: "location")
    basepath : str
        if set, use custom base path instead of output path
    for_gdal : bool
        use GDAL compatible remote paths, i.e. add "/vsicurl/" before path
        (default: True)


Return raster metadata.


Example process for testing.

    Inputs:
    -------
    file1
        raster file

    Parameters:
    -----------

    Output:
    -------
    np.ndarray


Check if output format is valid with other process parameters.

        Parameters
        ----------
        config : dictionary
            output configuration parameters

        Returns
        -------
        is_valid : bool


Return all available output formats.

    Returns
    -------
    formats : list
        all available output formats


Return output class of driver.

    Returns
    -------
    output : ``OutputData``
        output writer object


Return input class of driver.

    Returns
    -------
    input_params : ``InputData``
        input parameters


Dump output JSON and verify parameters if output metadata exist.


Determine target file path.

        Parameters
        ----------
        tile : ``BufferedTile``
            must be member of output ``TilePyramid``

        Returns
        -------
        path : string


Create directory and subdirectory if necessary.

        Parameters
        ----------
        tile : ``BufferedTile``
            must be member of output ``TilePyramid``


Check whether process output is allowed with output driver.

        Parameters
        ----------
        process_data : raw process output

        Returns
        -------
        True or False


Return verified and cleaned output.

        Parameters
        ----------
        process_data : raw process output

        Returns
        -------
        NumPy array or list of features.


Extract subset from multiple tiles.

        input_data_tiles : list of (``Tile``, process data) tuples
        out_tile : ``Tile``

        Returns
        -------
        NumPy array or list of features.


Calculate slope and aspect map.

    Return a pair of arrays 2 pixels smaller than the input elevation array.

    Slope is returned in radians, from 0 for sheer face to pi/2 for
    flat ground. Aspect is returned in radians, counterclockwise from -pi
    at north around to pi.

    Logic here is borrowed from hillshade.cpp:
    http://www.perrygeo.net/wordpress/?p=7

    Parameters
    ----------
    elevation : array
        input elevation data
    xres : float
        column width
    yres : float
        row  height
    z : float
        vertical exaggeration factor
    scale : float
        scale factor of pixel size units versus height units (insert 112000
        when having elevation values in meters in a geodetic projection)

    Returns
    -------
    slope shade : array


Return hillshaded numpy array.

    Parameters
    ----------
    elevation : array
        input elevation data
    tile : Tile
        tile covering the array
    z : float
        vertical exaggeration factor
    scale : float
        scale factor of pixel size units versus height units (insert 112000
        when having elevation values in meters in a geodetic projection)


Return ``BufferedTile`` object of this ``BufferedTilePyramid``.

        Parameters
        ----------
        zoom : integer
            zoom level
        row : integer
            tile matrix row
        col : integer
            tile matrix column

        Returns
        -------
        buffered tile : ``BufferedTile``


Return all tiles intersecting with bounds.

        Bounds values will be cleaned if they cross the antimeridian or are
        outside of the Northern or Southern tile pyramid bounds.

        Parameters
        ----------
        bounds : tuple
            (left, bottom, right, top) bounding values in tile pyramid CRS
        zoom : integer
            zoom level

        Yields
        ------
        intersecting tiles : generator
            generates ``BufferedTiles``


All metatiles intersecting with given bounding box.

        Parameters
        ----------
        geometry : ``shapely.geometry``
        zoom : integer
            zoom level

        Yields
        ------
        intersecting tiles : generator
            generates ``BufferedTiles``


Return all tiles intersecting with input geometry.

        Parameters
        ----------
        geometry : ``shapely.geometry``
        zoom : integer
            zoom level

        Yields
        ------
        intersecting tiles : ``BufferedTile``


Return all BufferedTiles intersecting with tile.

        Parameters
        ----------
        tile : ``BufferedTile``
            another tile


Return dictionary representation of pyramid parameters.


Return tile neighbors.

        Tile neighbors are unique, i.e. in some edge cases, where both the left
        and right neighbor wrapped around the antimeridian is the same. Also,
        neighbors ouside the northern and southern TilePyramid boundaries are
        excluded, because they are invalid.

        -------------
        | 8 | 1 | 5 |
        -------------
        | 4 | x | 2 |
        -------------
        | 7 | 3 | 6 |
        -------------

        Parameters
        ----------
        connectedness : int
            [4 or 8] return four direct neighbors or all eight.

        Returns
        -------
        list of BufferedTiles


Read, stretch and return raster data.

    Inputs:
    -------
    raster
        raster file

    Parameters:
    -----------
    resampling : str
        rasterio.Resampling method
    scale_method : str
        - dtype_scale: use dtype minimum and maximum values
        - minmax_scale: use dataset bands minimum and maximum values
        - crop: clip data to output dtype
    scales_minmax : tuple
        tuple of band specific scale values

    Output:
    -------
    np.ndarray


Open process output as input for other process.

        Parameters
        ----------
        tile : ``Tile``
        process : ``MapcheteProcess``
        kwargs : keyword arguments


Serve a Mapchete process.

    Creates the Mapchete host and serves both web page with OpenLayers and the
    WMTS simple REST endpoint.


Extract a numpy array from a raster file.


Extract raster data window array.

    Parameters
    ----------
    in_raster : array or ReferencedRaster
    in_affine : ``Affine`` required if in_raster is an array
    out_tile : ``BufferedTile``

    Returns
    -------
    extracted array : array


Extract and resample from array to target tile.

    Parameters
    ----------
    in_raster : array
    in_affine : ``Affine``
    out_tile : ``BufferedTile``
    resampling : string
        one of rasterio's resampling methods (default: nearest)
    nodataval : integer or float
        raster nodata value (default: 0)

    Returns
    -------
    resampled array : array


Determine if distance over antimeridian is shorter than normal distance.


Turn input data into a proper array for further usage.

    Outut array is always 3-dimensional with the given data type. If the output
    is masked, the fill_value corresponds to the given nodata value and the
    nodata value will be burned into the data array.

    Parameters
    ----------
    data : array or iterable
        array (masked or normal) or iterable containing arrays
    nodata : integer or float
        nodata value (default: 0) used if input is not a masked array and
        for output array
    masked : bool
        return a NumPy Array or a NumPy MaskedArray (default: True)
    dtype : string
        data type of output array (default: "int16")

    Returns
    -------
    array : array


Reproject a geometry to target CRS.

    Also, clips geometry if it lies outside the destination CRS boundary.
    Supported destination CRSes for clipping: 4326 (WGS84), 3857 (Spherical
    Mercator) and 3035 (ETRS89 / ETRS-LAEA).

    Parameters
    ----------
    geometry : ``shapely.geometry``
    src_crs : ``rasterio.crs.CRS`` or EPSG code
        CRS of source data
    dst_crs : ``rasterio.crs.CRS`` or EPSG code
        target CRS
    error_on_clip : bool
        raises a ``RuntimeError`` if a geometry is outside of CRS bounds
        (default: False)
    validity_check : bool
        checks if reprojected geometry is valid and throws ``TopologicalError``
        if invalid (default: True)
    antimeridian_cutting : bool
        cut geometry at Antimeridian; can result in a multipart output geometry

    Returns
    -------
    geometry : ``shapely.geometry``


Segmentize Polygon outer ring by segmentize value.

    Just Polygon geometry type supported.

    Parameters
    ----------
    geometry : ``shapely.geometry``
    segmentize_value: float

    Returns
    -------
    geometry : ``shapely.geometry``


Write features to GeoJSON file.

    Parameters
    ----------
    in_data : features
    out_schema : dictionary
        output schema for fiona
    out_tile : ``BufferedTile``
        tile used for output extent
    out_path : string
        output path for GeoJSON file


Return geometry of a specific type if possible.

    Filters and splits up GeometryCollection into target types. This is
    necessary when after clipping and/or reprojecting the geometry types from
    source geometries change (i.e. a Polygon becomes a LineString or a
    LineString becomes Point) in some edge cases.

    Parameters
    ----------
    geometry : ``shapely.geometry``
    target_type : string
        target geometry type
    allow_multipart : bool
        allow multipart geometries (default: True)

    Returns
    -------
    cleaned geometry : ``shapely.geometry``
        returns None if input geometry type differs from target type

    Raises
    ------
    GeometryTypeError : if geometry type does not match target_type


Yield single part geometries if geom is multipart, otherwise yield geom.

    Parameters:
    -----------
    geom : shapely geometry

    Returns:
    --------
    shapely single part geometries


Convert and optionally clip input raster data.

    Inputs:
    -------
    raster
        singleband or multiband data input
    clip (optional)
        vector data used to clip output

    Parameters
    ----------
    td_resampling : str (default: 'nearest')
        Resampling used when reading from TileDirectory.
    td_matching_method : str ('gdal' or 'min') (default: 'gdal')
        gdal: Uses GDAL's standard method. Here, the target resolution is
            calculated by averaging the extent's pixel sizes over both x and y
            axes. This approach returns a zoom level which may not have the
            best quality but will speed up reading significantly.
        min: Returns the zoom level which matches the minimum resolution of the
            extents four corner pixels. This approach returns the zoom level
            with the best possible quality but with low performance. If the
            tile extent is outside of the destination pyramid, a
            TopologicalError will be raised.
    td_matching_max_zoom : int (optional, default: None)
        If set, it will prevent reading from zoom levels above the maximum.
    td_matching_precision : int (default: 8)
        Round resolutions to n digits before comparing.
    td_fallback_to_higher_zoom : bool (default: False)
        In case no data is found at zoom level, try to read data from higher
        zoom levels. Enabling this setting can lead to many IO requests in
        areas with no data.
    clip_pixelbuffer : int
        Use pixelbuffer when clipping output by geometry. (default: 0)

    Output
    ------
    np.ndarray


Determine the best base zoom level for a raster.

    "Best" means the maximum zoom level where no oversampling has to be done.

    Parameters
    ----------
    input_file : path to raster file
    tile_pyramid_type : ``TilePyramid`` projection (``geodetic`` or``mercator``)

    Returns
    -------
    zoom : integer


Determine whether file path is remote or local.

    Parameters
    ----------
    path : path to file

    Returns
    -------
    is_remote : bool


Check if file exists either remote or local.

    Parameters:
    -----------
    path : path to file

    Returns:
    --------
    exists : bool


Return absolute path if path is local.

    Parameters:
    -----------
    path : path to file
    base_dir : base directory used for absolute path

    Returns:
    --------
    absolute path


Return relative path if path is local.

    Parameters:
    -----------
    path : path to file
    base_dir : directory where path sould be relative to

    Returns:
    --------
    relative path


Write local or remote.


Read local or remote.


Attach a reducer function to a given type in the dispatch table.


Return the number of CPUs the current process can use.

    The returned number of CPUs accounts for:
     * the number of CPUs in the system, as given by
       ``multiprocessing.cpu_count``;
     * the CPU affinity settings of the current process
       (available with Python 3.4+ on some Unix systems);
     * CFS scheduler CPU bandwidth limit (available on Linux only, typically
       set by docker and similar container orchestration systems);
     * the value of the LOKY_MAX_CPU_COUNT environment variable if defined.
    and is given as the minimum of these constraints.
    It is also always larger or equal to 1.


Evaluates calls from call_queue and places the results in result_queue.

    This worker is run in a separate process.

    Args:
        call_queue: A ctx.Queue of _CallItems that will be read and
            evaluated by the worker.
        result_queue: A ctx.Queue of _ResultItems that will written
            to by the worker.
        initializer: A callable initializer, or None
        initargs: A tuple of args for the initializer
        process_management_lock: A ctx.Lock avoiding worker timeout while some
            workers are being spawned.
        timeout: maximum time to wait for a new item in the call_queue. If that
            time is expired, the worker will shutdown.
        worker_exit_lock: Lock to avoid flagging the executor as broken on
            workers timeout.
        current_depth: Nested parallelism level, to avoid infinite spawning.


Fills call_queue with _WorkItems from pending_work_items.

    This function never blocks.

    Args:
        pending_work_items: A dict mapping work ids to _WorkItems e.g.
            {5: <_WorkItem...>, 6: <_WorkItem...>, ...}
        work_ids: A queue.Queue of work ids e.g. Queue([5, 6, ...]). Work ids
            are consumed and the corresponding _WorkItems from
            pending_work_items are transformed into _CallItems and put in
            call_queue.
        call_queue: A ctx.Queue that will be filled with _CallItems
            derived from _WorkItems.


Wrapper for non-picklable object to use cloudpickle to serialize them.

    Note that this wrapper tends to slow down the serialization process as it
    is done with cloudpickle which is typically slower compared to pickle. The
    proper way to solve serialization issues is to avoid defining functions and
    objects in the main scripts and to implement __reduce__ functions for
    complex classes.


Return a wrapper for an fd.


Return the current ReusableExectutor instance.

    Start a new instance if it has not been started already or if the previous
    instance was left in a broken state.

    If the previous instance does not have the requested number of workers, the
    executor is dynamically resized to adjust the number of workers prior to
    returning.

    Reusing a singleton instance spares the overhead of starting new worker
    processes and importing common python packages each time.

    ``max_workers`` controls the maximum number of tasks that can be running in
    parallel in worker processes. By default this is set to the number of
    CPUs on the host.

    Setting ``timeout`` (in seconds) makes idle workers automatically shutdown
    so as to release system resources. New workers are respawn upon submission
    of new tasks so that ``max_workers`` are available to accept the newly
    submitted tasks. Setting ``timeout`` to around 100 times the time required
    to spawn new processes and import packages in them (on the order of 100ms)
    ensures that the overhead of spawning workers is negligible.

    Setting ``kill_workers=True`` makes it possible to forcibly interrupt
    previously spawned jobs to get a new instance of the reusable executor
    with new constructor argument values.

    The ``job_reducers`` and ``result_reducers`` are used to customize the
    pickling of tasks and results send to the executor.

    When provided, the ``initializer`` is run first in newly spawned
    processes with argument ``initargs``.


Wait for the cache to be empty before resizing the pool.


Return info about parent needed by child to unpickle process object


Try to get current process ready to unpickle process object


Close all the file descriptors except those in keep_fds.


Return a formated string with the exitcodes of terminated workers.

    If necessary, wait (up to .25s) for the system to correctly set the
    exitcode of one terminated worker.


Format a list of exit code with names of the signals if possible


Run semaphore tracker.


Make sure that semaphore tracker process is running.

        This can be run from any process.  Usually a child process will use
        the semaphore created by its parent.


A simple event processor that prints out events.


Program counter.


Almost a copy of code.interact
    Closely emulate the interactive Python interpreter.

    This is a backwards compatible interface to the InteractiveConsole
    class.  When readfunc is not specified, it attempts to import the
    readline module to enable GNU readline if it is available.

    Arguments (all optional, all default to None):

    banner -- passed to InteractiveConsole.interact()
    readfunc -- if not None, replaces InteractiveConsole.raw_input()
    local -- passed to InteractiveInterpreter.__init__()


Split a command line's arguments in a shell-like manner returned
    as a list of lists. Use ';;' with white space to indicate separate
    commands.

    This is a modified version of the standard library's shlex.split()
    function, but with a default of posix=False for splitting, so that quotes
    in inputs are respected.


Run each function in `hooks' with args


Eval arg and it is an integer return the value. Otherwise
        return None


If no argument use the default. If arg is a an integer between
        least min_value and at_most, use that. Otherwise report an error.
        If there's a stack frame use that in evaluation.


Find the next token in str string from start_pos, we return
    the token and the next blank position after the token or
    str.size if this is the last token. Tokens are delimited by
    white space.


Script interface to read a command. `prompt' is a parameter for
        compatibilty and is ignored.


Closes both input and output


Disassemble byte string of code. If end_line is negative
    it counts the number of statement linestarts to use.


Return a count of the number of frames


If f_back is looking at a call function, return
    the name for it. Otherwise return None


Print count entries of the stack trace


Find subcmd in self.subcmds


Show short help for a subcommand.


Add subcmd to the available subcommands for this object.
        It will have the supplied docstring, and subcmd_cb will be called
        when we want to run the command. min_len is the minimum length
        allowed to abbreviate the command. in_list indicates with the
        show command will be run when giving a list of all sub commands
        of this object. Some commands have long output like "show commands"
        so we might not want to show that.


Run subcmd_name with args using obj for the environent


Enter the debugger.

Parameters
----------

level : how many stack frames go back. Usually it will be
the default 0. But sometimes though there may be calls in setup to the debugger
that you may want to skip.

step_ignore : how many line events to ignore after the
debug() call. 0 means don't even wait for the debug() call to finish.

param dbg_opts : is an optional "options" dictionary that gets fed
trepan.Debugger(); `start_opts' are the optional "options"
dictionary that gets fed to trepan.Debugger.core.start().

Use like this:

.. code-block:: python

    ... # Possibly some Python code
    import trepan.api # Needed only once
    ... # Possibly some more Python code
    trepan.api.debug() # You can wrap inside conditional logic too
    pass  # Stop will be here.
    # Below is code you want to use the debugger to do things.
    ....  # more Python code
    # If you get to a place in the program where you aren't going
    # want to debug any more, but want to remove debugger trace overhead:
    trepan.api.stop()

Parameter "level" specifies how many stack frames go back. Usually it will be
the default 0. But sometimes though there may be calls in setup to the debugger
that you may want to skip.

Parameter "step_ignore" specifies how many line events to ignore after the
debug() call. 0 means don't even wait for the debug() call to finish.

In situations where you want an immediate stop in the "debug" call
rather than the statement following it ("pass" above), add parameter
step_ignore=0 to debug() like this::

    import trepan.api  # Needed only once
    # ... as before
    trepan.api.debug(step_ignore=0)
    # ... as before

Module variable _debugger_obj_ from module trepan.debugger is used as
the debugger instance variable; it can be subsequently used to change
settings or alter behavior. It should be of type Debugger (found in
module trepan). If not, it will get changed to that type::

   $ python
   >>> from trepan.debugger import debugger_obj
   >>> type(debugger_obj)
   <type 'NoneType'>
   >>> import trepan.api
   >>> trepan.api.debug()
   ...
   (Trepan) c
   >>> from trepan.debugger import debugger_obj
   >>> debugger_obj
   <trepan.debugger.Debugger instance at 0x7fbcacd514d0>
   >>>

If however you want your own separate debugger instance, you can
create it from the debugger _class Debugger()_ from module
trepan.debugger::

  $ python
  >>> from trepan.debugger import Debugger
  >>> dbgr = Debugger()  # Add options as desired
  >>> dbgr
  <trepan.debugger.Debugger instance at 0x2e25320>

`dbg_opts' is an optional "options" dictionary that gets fed
trepan.Debugger(); `start_opts' are the optional "options"
dictionary that gets fed to trepan.Debugger.core.start().


Find the first frame that is a debugged frame. We do this
    Generally we want traceback information without polluting it with
    debugger frames. We can tell these because those are frames on the
    top which don't have f_trace set. So we'll look back from the top
    to find the fist frame where f_trace is set.


If arg is an int, use that otherwise take default.


Return True if arg is 'on' or 1 and False arg is 'off' or 0.
    Any other value is raises ValueError.


set a Boolean-valued debugger setting. 'obj' is a generally a
    subcommand that has 'name' and 'debugger.settings' attributes


set an Integer-valued debugger setting. 'obj' is a generally a
    subcommand that has 'name' and 'debugger.settings' attributes


Generic subcommand showing a boolean-valued debugger setting.
    'obj' is generally a subcommand that has 'name' and
    'debugger.setting' attributes.


Return True if we are looking at a def statement


Get bacground from
    default values based on the TERM environment variable


Pass as parameters R G B values in hex
    On return, variable is_dark_bg is set


return suitable frame signature to key display expressions off of.


display any items that are active


Set breakpoint at current location, or a specified frame


Find the corresponding signal name for 'num'. Return None
    if 'num' is invalid.


Find the corresponding signal number for 'name'. Return None
    if 'name' is invalid.


Return a signal name for a signal name or signal
    number.  Return None is name_num is an int but not a valid signal
    number and False if name_num is a not number. If name_num is a
    signal name or signal number, the canonic if name is returned.


Check to see if any of the signal handlers we are interested in have
        changed or is not initially set. Change any that are not right.


Print information about a signal


Delegate the actions specified in 'arg' to another
        method.


Return a full pathname for filename if we can find one. path
    is a list of directories to prepend to filename. If no file is
    found we'll return None


Do a shell-like path lookup for py_script and return the results.
    If we can't find anything return py_script


used to write to a debugger that is connected to this
        server; `str' written will have a newline added to it


Execution status of the program.


List commands arranged in an aligned columns


Enter debugger read loop after your program has crashed.

    exc is a triple like you get back from sys.exc_info.  If no exc
    parameter, is supplied, the values from sys.last_type,
    sys.last_value, sys.last_traceback are used. And if these don't
    exist either we'll assume that sys.exc_info() contains what we
    want and frameno is the index location of where we want to start.

    'frameno' specifies how many frames to ignore in the traceback.
    The default is 1, that is, we don't need to show the immediate
    call into post_mortem. If you have wrapper functions that call
    this one, you may want to increase frameno.


Closes both socket and server connection.


This method the debugger uses to write. In contrast to
        writeline, no newline is added to the end to `str'. Also
        msg doesn't have to be a string.


Complete an arbitrary expression.


Add `frame_or_fn' to the list of functions that are not to
        be debugged


Turns `filename' into its canonic representation and returns this
        string. This allows a user to refer to a given file in one of several
        equivalent ways.

        Relative filenames need to be fully resolved, since the current working
        directory might change over the course of execution.

        If filename is enclosed in < ... >, then we assume it is
        one of the bogus internal Python names like <string> which is seen
        for example when executing "exec cmd".


Return filename or the basename of that depending on the
        basename setting


Return True if debugging is in progress.


Does the magic to determine if we stop here and run a
        command processor or not. If so, return True and set
        self.stop_reason; if not, return False.

        Determining factors can be whether a breakpoint was
        encountered, whether we are stepping, next'ing, finish'ing,
        and, if so, whether there is an ignore counter.


Sets to stop on the next event that happens in frame 'frame'.


A mini stack trace routine for threads.


Get file information


Check whether we should break here because of `b.funcname`.


remove breakpoint `bp


Remove a breakpoint given its breakpoint number.


Enable or disable all breakpoints.


Enable or disable a breakpoint given its breakpoint number.


Read a line of input. Prompt and use_raw exist to be
        compatible with other input routines and are ignored.
        EOFError will be raised on EOF.


Restore an original login session, checking the signed session


Yield each document in a Luminoso project in turn. Requires a client whose
    URL points to a project.

    If expanded=True, it will include additional fields that Luminoso added in
    its analysis, such as 'terms' and 'vector'.

    Otherwise, it will contain only the fields necessary to reconstruct the
    document: 'title', 'text', and 'metadata'.

    Shows a progress bar if progress=True.


Handle arguments for the 'lumi-download' command.


Read a JSON or CSV file and convert it into a JSON stream, which will
    be saved in an anonymous temp file.


Deduce the format of a file, within reason.

    - If the filename ends with .csv or .txt, it's csv.
    - If the filename ends with .jsons, it's a JSON stream (conveniently the
      format we want to output).
    - If the filename ends with .json, it could be a legitimate JSON file, or
      it could be a JSON stream, following a nonstandard convention that many
      people including us are guilty of. In that case:
      - If the first line is a complete JSON document, and there is more in the
        file besides the first line, then it is a JSON stream.
      - Otherwise, it is probably really JSON.
    - If the filename does not end with .json, .jsons, or .csv, we have to guess
      whether it's still CSV or tab-separated values or something like that.
      If it's JSON, the first character would almost certainly have to be a
      bracket or a brace. If it isn't, assume it's CSV or similar.


This function is meant to normalize data for upload to the Luminoso
    Analytics system. Currently it only normalizes dates.

    If date_format is not specified, or if there's no date in a particular doc,
    the the doc is yielded unchanged.


Convert a date in a given format to epoch time. Mostly a wrapper for
    datetime's strptime.


Open a CSV file using Python 2's CSV module, working around the deficiency
    where it can't handle the null bytes of UTF-16.


Handle command line arguments to convert a file to a JSON stream as a
    script.


Returns an object that makes requests to the API, authenticated
        with a saved or specified long-lived token, at URLs beginning with
        `url`.

        If no URL is specified, or if the specified URL is a path such as
        '/projects' without a scheme and domain, the client will default to
        https://analytics.luminoso.com/api/v5/.

        If neither token nor token_file are specified, the client will look
        for a token in $HOME/.luminoso/tokens.json. The file should contain
        a single json dictionary of the format
        `{'root_url': 'token', 'root_url2': 'token2', ...}`.


Take a long-lived API token and store it to a local file.  Long-lived
        tokens can be retrieved through the UI.  Optional arguments are the
        domain for which the token is valid and the file in which to store the
        token.


Make a DELETE request to the given path, and return the JSON-decoded
        result.

        Keyword parameters will be converted to URL parameters.

        DELETE requests ask to delete the object represented by this URL.


A convenience method designed to inform you when a project build has
        completed.  It polls the API every `interval` seconds until there is
        not a build running.  At that point, it returns the "last_build_info"
        field of the project record if the build succeeded, and raises a
        LuminosoError with the field as its message if the build failed.

        If a `path` is not specified, this method will assume that its URL is
        the URL for the project.  Otherwise, it will use the specified path
        (which should be "/projects/<project_id>/").


Get the "root URL" for a URL, as described in the LuminosoClient
    documentation.


Obtain the user's long-lived API token and save it in a local file.
        If the user has no long-lived API token, one will be created.
        Returns the token that was saved.


Make a request of the specified type and expect a JSON object in
        response.

        If the result has an 'error' value, raise a LuminosoAPIError with
        its contents. Otherwise, return the contents of the 'result' value.


Get the ID of an account you can use to access projects.


Get the documentation that the server sends for the API.


Wait for an asynchronous task to finish.

        Unlike the thin methods elsewhere on this object, this one is actually
        specific to how the Luminoso API works. This will poll an API
        endpoint to find out the status of the job numbered `job_id`,
        repeating every 5 seconds (by default) until the job is done. When
        the job is done, it will return an object representing the result of
        that job.

        In the Luminoso API, requests that may take a long time return a
        job ID instead of a result, so that your code can continue running
        in the meantime. When it needs the job to be done to proceed, it can
        use this method to wait.

        The base URL where it looks for that job is by default `jobs/id/`
        under the current URL, assuming that this LuminosoClient's URL
        represents a project. You can specify a different URL by changing
        `base_path`.

        If the job failed, will raise a LuminosoError with the job status
        as its message.


Get the raw text of a response.

        This is only generally useful for specific URLs, such as documentation.


Print a JSON list of JSON objects in CSV format.


Read parameters from input file, -j, and -p arguments, in that order.


Limit a document to just the three fields we should upload.


Given an iterator of documents, upload them as a Luminoso project.


Handle arguments for the 'lumi-upload' command.


Upload a file to Luminoso with the given account and project name.

    Given a file containing JSON, JSON stream, or CSV data, this verifies
    that we can successfully convert it to a JSON stream, then uploads that
    JSON stream.


Handle command line arguments, to upload a file to a Luminoso project
    as a script.


Obtain a short-lived token using a username and password, and use that
        token to create an auth object.


Set http session.


Login to enedis.


Get data.


Get the latest data from Enedis.


Load the view on first load


Load the view on first load could also load based on session, group, etc..


Execute the correct handler depending on what is connecting.


When enaml.js sends a message


When pages change, update the menus


Generate the handlers for this site


Create the toolkit widget for the proxy object.

        This method is called during the top-down pass, just before the
        'init_widget()' method is called. This method should create the
        toolkit widget and assign it to the 'widget' attribute.


Initialize the state of the toolkit widget.

        This method is called during the top-down pass, just after the
        'create_widget()' method is called. This method should init the
        state of the widget. The child widgets will not yet be created.


A reimplemented destructor.

        This destructor will clear the reference to the toolkit widget
        and set its parent to None.


Handle the child added event from the declaration.

        This handler will insert the child toolkit widget in the correct.
        position. Subclasses which need more control should reimplement this
        method.


Handle the child removed event from the declaration.

        This handler will unparent the child toolkit widget. Subclasses
        which need more control should reimplement this method.


Default handler for those not explicitly defined


Update the proxy widget when the Widget data
        changes.


Find nodes matching the given xpath query


Initialize the widget with the source.


A change handler for the 'objects' list of the Include.

        If the object is initialized objects which are removed will be
        unparented and objects which are added will be reparented. Old
        objects will be destroyed if the 'destroy_old' flag is True.


When the children of the block change. Update the referenced
        block.


Registers a function as a hook. Multiple hooks can be registered for a given type, but the
        order in which they are invoke is unspecified.

        :param event_type: The event type this hook will be invoked for.


Callback from Flask


Remove common indentation from string.

    Unlike doctrim there is no special treatment of the first line.


Find all section names and return a list with their names.


Generate table of contents for array of section names.


Print `msg` error and exit with status `exit_code`


Gets a Item from the Menu by name. Note that the name is not
        case-sensitive but must be spelt correctly.

        :param string name: The name of the item.
        :raises StopIteration: Raises exception if no item is found.
        :return: An item object matching the search.
        :rtype: Item


Clear out the current session on the remote and setup a new one.

        :return: A response from having expired the current session.
        :rtype: requests.Response


Search for dominos pizza stores using a search term.

        :param string search: Search term.
        :return: A list of nearby stores matching the search term.
        :rtype: list


Add an item to the current basket.

        :param Item item: Item from menu.
        :param int variant: Item SKU id. Ignored if the item is a side.
        :param int quantity: The quantity of item to be added.
        :return: A response having added an item to the current basket.
        :rtype: requests.Response


Add a pizza to the current basket.

        :param Item item: Item from menu.
        :param int variant: Item SKU id. Some defaults are defined in the VARIANT enum.
        :param int quantity: The quantity of pizza to be added.
        :return: A response having added a pizza to the current basket.
        :rtype: requests.Response


Add a side to the current basket.

        :param Item item: Item from menu.
        :param int quantity: The quantity of side to be added.
        :return: A response having added a side to the current basket.
        :rtype: requests.Response


Remove an item from the current basket.

        :param int idx: Basket item id.
        :return: A response having removed an item from the current basket.
        :rtype: requests.Response


Select the payment method going to be used to make a purchase.

        :param int method: Payment method id.
        :return: A response having set the payment option.
        :rtype: requests.Response


Proceed with payment using the payment method selected earlier.

        :return: A response having processes the payment.
        :rtype: requests.Response


Make a HTTP GET request to the Dominos UK API with the given parameters
        for the current session.

        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response


Make a HTTP POST request to the Dominos UK API with the given
        parameters for the current session.

        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response


Make a HTTP request to the Dominos UK API with the given parameters for
        the current session.

        :param verb func: HTTP method on the session.
        :param string path: The API endpoint path.
        :params list kargs: A list of arguments.
        :return: A response from the Dominos UK API.
        :rtype: response.Response


Add an item to the end of the menu before the exit item

        :param MenuItem item: The item to be added


Add the exit item if necessary. Used to make sure there aren't multiple exit items

        :return: True if item needed to be added, False otherwise
        :rtype: bool


Redraws the menu and refreshes the screen. Should be called whenever something changes that needs to be redrawn.


Gets the next single character and decides what to do with it


Select the current item and run it


Take an old-style menuData dictionary and return a CursesMenu

    :param dict menu_data:
    :return: A new CursesMenu
    :rtype: CursesMenu


Compute the maximum temporal distance.

        Returns
        -------
        max_temporal_distance : float


Temporal distance cumulative density function.

        Returns
        -------
        x_values: numpy.array
            values for the x-axis
        cdf: numpy.array
            cdf values


Remove dangling entries from the shapes directory.

    Parameters
    ----------
    db_conn: sqlite3.Connection
        connection to the GTFS object


Given a set of transit events and the static walk network,
    "transform" the static walking network into a set of "pseudo-connections".

    As a first approximation, we add pseudo-connections to depart after each arrival of a transit connection
    to it's arrival stop.

    Parameters
    ----------
    transit_connections: list[Connection]
    start_time_dep : int
        start time in unixtime seconds
    end_time_dep: int
        end time in unixtime seconds (no new connections will be scanned after this time)
    transfer_margin: int
        required extra margin required for transfers in seconds
    walk_speed: float
        walking speed between stops in meters / second
    walk_network: networkx.Graph
        each edge should have the walking distance as a data attribute ("d_walk") expressed in meters

    Returns
    -------
    pseudo_connections: set[Connection]


Get the earliest visit time of the stop.


Whether the spreading stop can infect using this event.


Create day_trips and day_stop_times views.

        day_trips:  day_trips2 x trips  = days x trips
        day_stop_times: day_trips2 x trips x stop_times = days x trips x stop_times


Create a colourbar with limits of lwr and upr


Write temporal networks by route type to disk.

    Parameters
    ----------
    gtfs: gtfspy.GTFS
    extract_output_dir: str


Write out the database according to the GTFS format.

    Parameters
    ----------
    gtfs: gtfspy.GTFS
    output: str
        Path where to put the GTFS files
        if output ends with ".zip" a ZIP-file is created instead.

    Returns
    -------
    None


Remove columns ending with I from a pandas.DataFrame

    Parameters
    ----------
    df: dataFrame

    Returns
    -------
    None


Context manager for making files with possibility of failure.

    If you are creating a file, it is possible that the code will fail
    and leave a corrupt intermediate file.  This is especially damaging
    if this is used as automatic input to another process.  This context
    manager helps by creating a temporary filename, your code runs and
    creates that temporary file, and then if no exceptions are raised,
    the context manager will move the temporary file to the original
    filename you intended to open.

    Parameters
    ----------
    fname : str
        Target filename, this file will be created if all goes well
    fname_tmp : str
        If given, this is used as the temporary filename.
    tmpdir : str or bool
        If given, put temporary files in this directory.  If `True`,
        then find a good tmpdir that is not on local filesystem.
    save_tmpfile : bool
        If true, the temporary file is not deleteted if an exception
        is raised.
    keepext : bool, default False
            If true, have tmpfile have same extension as final file.

    Returns (as context manager value)
    ----------------------------------
     fname_tmp: str
        Temporary filename to be used.  Same as `fname_tmp`
        if given as an argument.

    Raises
    ------
    Re-raises any except occuring during the context block.


Utility function to print sqlite queries before executing.

    Use instead of cur.execute().  First argument is cursor.

    cur.execute(stmt)
    becomes
    util.execute(cur, stmt)


Create directories if they do not exist, otherwise do nothing.

    Return path for convenience


Checks for rows that are not referenced in the the tables that should be linked

        stops <> stop_times using stop_I
        stop_times <> trips <> days, using trip_I
        trips <> routes, using route_I
        :return:


Print coordinates within a sequence.

    This is only used for debugging.  Printed in a form that can be
    pasted into Python for visualization.


Find corresponding shape points for a list of stops and create shape break points.

    Parameters
    ----------
    stops: stop-sequence (list)
        List of stop points
    shape: list of shape points
        shape-sequence of shape points

    Returns
    -------
    break_points: list[int]
        stops[i] corresponds to shape[break_points[i]].  This list can
        be used to partition the shape points into segments between
        one stop and the next.
    badness: float
        Lower indicates better fit to the shape.  This is the sum of
        distances (in meters) between every each stop and its closest
        shape point.  This is not needed in normal use, but in the
        cases where you must determine the best-fitting shape for a
        stop-sequence, use this.


Get all scheduled stops on a particular route_id.

    Given a route_id, return the trip-stop-list with
    latitude/longitudes.  This is a bit more tricky than it seems,
    because we have to go from table route->trips->stop_times.  This
    functions finds an arbitrary trip (in trip table) with this route ID
    and, and then returns all stop points for that trip.

    Parameters
    ----------
    cur : sqlite3.Cursor
        cursor to sqlite3 DB containing GTFS
    route_id : string or any
        route_id to get stop points of
    offset : int
        LIMIT offset if you don't want the first trip returned.
    tripid_glob : string
        If given, allows you to limit tripids which can be selected.
        Mainly useful in debugging.

    Returns
    -------
    stop-list
        List of stops in stop-seq format.


Interpolate passage times for shape points.

    Parameters
    ----------
    shape_distances: list
        list of cumulative distances along the shape
    shape_breaks: list
        list of shape_breaks
    stop_times: list
        list of stop_times

    Returns
    -------
    shape_times: list of ints (seconds) / numpy array
        interpolated shape passage times

    The values of stop times before the first shape-break are given the first
    stopping time, and the any shape points after the last break point are
    given the value of the last shape point.


Get the earliest arrival time at the target, given a departure time.

        Parameters
        ----------
        dep_time : float, int
            time in unix seconds
        transfer_margin: float, int
            transfer margin in seconds

        Returns
        -------
        arrival_time : float
            Arrival time in the given time unit (seconds after unix epoch).


Get a stop-to-stop network describing a single mode of travel.

    Parameters
    ----------
    gtfs : gtfspy.GTFS
    route_type : int
        See gtfspy.route_types.TRANSIT_ROUTE_TYPES for the list of possible types.
    link_attributes: list[str], optional
        defaulting to use the following link attributes:
            "n_vehicles" : Number of vehicles passed
            "duration_min" : minimum travel time between stops
            "duration_max" : maximum travel time between stops
            "duration_median" : median travel time between stops
            "duration_avg" : average travel time between stops
            "d" : distance along straight line (wgs84_distance)
            "distance_shape" : minimum distance along shape
            "capacity_estimate" : approximate capacity passed through the stop
            "route_I_counts" : dict from route_I to counts
    start_time_ut: int
        start time of the time span (in unix time)
    end_time_ut: int
        end time of the time span (in unix time)

    Returns
    -------
    net: networkx.DiGraph
        A directed graph Directed graph


Compute stop-to-stop networks for all travel modes and combine them into a single network.
    The modes of transport are encoded to a single network.
    The network consists of multiple links corresponding to each travel mode.
    Walk mode is not included.

    Parameters
    ----------
    gtfs: gtfspy.GTFS

    Returns
    -------
    net: networkx.MultiDiGraph
        keys should be one of route_types.TRANSIT_ROUTE_TYPES (i.e. GTFS route_types)


Compute the temporal network of the data, and return it as a pandas.DataFrame

    Parameters
    ----------
    gtfs : gtfspy.GTFS
    start_time_ut: int | None
        start time of the time span (in unix time)
    end_time_ut: int | None
        end time of the time span (in unix time)
    route_type: int | None
        Specifies which mode of public transport are included, or whether all modes should be included.
        The int should be one of the standard GTFS route_types:
        (see also gtfspy.route_types.TRANSIT_ROUTE_TYPES )
        If route_type is not specified, all modes are included.

    Returns
    -------
    events_df: pandas.DataFrame
        Columns: departure_stop, arrival_stop, departure_time_ut, arrival_time_ut, route_type, route_I, trip_I


Get stop pairs through which transfers take place

        Returns
        -------
        transfer_stop_pairs: list


Get name of the GTFS timezone

        Returns
        -------
        timezone_name : str
            name of the time zone, e.g. "Europe/Helsinki"


Get the shapes of all routes.

        Parameters
        ----------
        use_shapes : bool, optional
            by default True (i.e. use shapes as the name of the function indicates)
            if False (fall back to lats and longitudes)

        Returns
        -------
        routeShapes: list of dicts that should have the following keys
            name, type, agency, lats, lons
            with types
            list, list, str, list, list


Get closest stop to a given location.

        Parameters
        ----------
        lat: float
            latitude coordinate of the location
        lon: float
            longitude coordinate of the location

        Returns
        -------
        stop_I: int
            the index of the stop in the database


Check that a trip takes place during a day

        Parameters
        ----------
        trip_I : int
            index of the trip in the gtfs data base
        day_start_ut : int
            the starting time of the day in unix time (seconds)

        Returns
        -------
        takes_place: bool
            boolean value describing whether the trip takes place during
            the given day or not


Get all possible day start times between start_ut and end_ut
        Currently this function is used only by get_tripIs_within_range_by_dsut

        Parameters
        ----------
        start_ut : list<int>
            start time in unix time
        end_ut : list<int>
            end time in unix time
        max_time_overnight : list<int>
            the maximum length of time that a trip can take place on
            during the next day (i.e. after midnight run times like 25:35)

        Returns
        -------
        day_start_times_ut : list
            list of ints (unix times in seconds) for returning all possible day
            start times
        start_times_ds : list
            list of ints (unix times in seconds) stating the valid start time in
            day seconds
        end_times_ds : list
            list of ints (unix times in seconds) stating the valid end times in
            day_seconds


Get all stop data as a pandas DataFrame for all stops, or an individual stop'

        Parameters
        ----------
        stop_I : int
            stop index

        Returns
        -------
        stop: pandas.DataFrame


Obtain a list of events that take place during a time interval.
        Each event needs to be only partially overlap the given time interval.
        Does not include walking events.

        Parameters
        ----------
        start_time_ut : int
            start of the time interval in unix time (seconds)
        end_time_ut: int
            end of the time interval in unix time (seconds)
        route_type: int
            consider only events for this route_type

        Returns
        -------
        events: pandas.DataFrame
            with the following columns and types
                dep_time_ut: int
                arr_time_ut: int
                from_stop_I: int
                to_stop_I: int
                trip_I : int
                shape_id : int
                route_type : int

        See also
        --------
        get_transit_events_in_time_span : an older version of the same thing


Return the first and last day_start_ut

        Returns
        -------
        first_day_start_ut: int
        last_day_start_ut: int


Recover pre-computed travel_impedance between od-pairs from the database.

        Returns
        -------
        values: number | Pandas DataFrame


Update the profile with the new labels.
        Each new label should have the same departure_time.

        Parameters
        ----------
        new_labels: list[LabelTime]

        Returns
        -------
        added: bool
            whether new_pareto_tuple was added to the set of pareto-optimal tuples


Get the pareto_optimal set of Labels, given a departure time.

        Parameters
        ----------
        dep_time : float, int
            time in unix seconds
        first_leg_can_be_walk : bool, optional
            whether to allow walking to target to be included into the profile
            (I.e. whether this function is called when scanning a pseudo-connection:
            "double" walks are not allowed.)
        connection_arrival_time: float, int, optional
            used for computing the walking label if dep_time, i.e., connection.arrival_stop_next_departure_time, is infinity)
        connection: connection object

        Returns
        -------
        pareto_optimal_labels : set
            Set of Labels


Do the actual import. Copy data and store in connection object.

        This function:
        - Creates the tables
        - Imports data (using self.gen_rows)
        - Run any post_import hooks.
        - Creates any indexs
        - Does *not* run self.make_views - those must be done
          after all tables are loaded.


Get mean latitude AND longitude of stops

    Parameters
    ----------
    gtfs: GTFS

    Returns
    -------
    mean_lat : float
    mean_lon : float


Writes data from get_stats to csv file

    Parameters
    ----------
    gtfs: GTFS
    path_to_csv: str
        filepath to the csv file to be generated
    re_write:
        insted of appending, create a new one.


Return the frequency of all types of routes per day.

    Parameters
    -----------
    gtfs: GTFS

    Returns
    -------
    pandas.DataFrame with columns
        route_I, type, frequency


A Python decorator for printing out the execution time for a function.

    Adapted from:
    www.andreas-jung.com/contents/a-python-decorator-for-measuring-the-execution-time-of-methods


When receiving the filled out form, check for valid access.


Return a form class for a given string pointing to a lockdown form.


Check if each request is allowed to access the current resource.


Handle redirects properly.


Get the top or flop N results based on a column value for each specified group columns

    ---

    ### Parameters

    *mandatory :*
    - `value` (*str*): column name on which you will rank the results
    - `limit` (*int*): Number to specify the N results you want to retrieve.
        Use a positive number x to retrieve the first x results.
        Use a negative number -x to retrieve the last x results.

    *optional :*
    - `order` (*str*): `"asc"` or `"desc"` to sort by ascending ou descending order. By default : `"asc"`.
    - `group` (*str*, *list of str*): name(s) of columns on which you want to perform the group operation.

    ---

    ### Example

    **Input**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lili   |    1     |  50  |
    |   lili   |    1     |  20  |
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |

    ```cson
    top:
      value: 'value'
      limit: 4
      order: 'asc'
    ```

    **Output**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lala   |    1     |  250  |
    |   toto   |    1     |  300  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |


Get the top or flop N results based on a function and a column value that agregates the input.
    The result is composed by all the original lines including only lines corresponding
    to the top groups

    ---

    ### Parameters

    *mandatory :*
    - `value` (*str*): Name of the column name on which you will rank the results.
    - `limit` (*int*): Number to specify the N results you want to retrieve from the sorted values.
        - Use a positive number x to retrieve the first x results.
        - Use a negative number -x to retrieve the last x results.
    - `aggregate_by` (*list of str*)): name(s) of columns you want to aggregate

    *optional :*
    - `order` (*str*): `"asc"` or `"desc"` to sort by ascending ou descending order. By default : `"asc"`.
    - `group` (*str*, *list of str*): name(s) of columns on which you want to perform the group operation.
    - `function` : Function to use to group over the group column

    ---

    ### Example

    **Input**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   lili   |    1     |  50  |
    |   lili   |    1     |  20  |
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |

    ```cson
    top_group:
      group: ["Category"]
      value: 'value'
      aggregate_by: ["variable"]
      limit: 2
      order: "desc"
    ```

    **Output**

    | variable | Category | value |
    |:--------:|:--------:|:-----:|
    |   toto   |    1     |  100  |
    |   toto   |    1     |  200  |
    |   toto   |    1     |  300  |
    |   lala   |    1     |  100  |
    |   lala   |    1     |  150  |
    |   lala   |    1     |  250  |
    |   lala   |    2     |  350  |
    |   lala   |    2     |  450  |


Convert string column into datetime column

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to format
    - `format` (*str*): current format of the values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))


Convert datetime column into string column

    ---

    ### Parameters

    *mandatory :*
    - column (*str*): name of the column to format
    - format (*str*): format of the result values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))

    *optional :*
    - new_column (*str*): name of the output column. By default `column` is overwritten.


Convert the format of a date

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to change the format
    - `output_format` (*str*): format of the output values (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior))

    *optional :*
    - `input_format` (*str*): format of the input values (by default let the parser detect it)
    - `new_column` (*str*): name of the output column  (by default overwrite `column`)
    - `new_time_zone` (*str*): name of new time zone (by default no time zone conversion is done)

    ---

    ### Example

    **Input**

    label   | date
    :------:|:----:
    France  | 2017-03-22
    Europe  | 2016-03-22

    ```cson
    change_date_format:
      column: 'date'
      input_format: '%Y-%m-%d'
      output_format: '%Y-%m'
    ```

    Output :

    label   | date
    :------:|:----:
    France  | 2017-03
    Europe  | 2016-03


Convert column's type into type

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column to convert
    - `type` (*str*): output type. It can be :
        - `"int"` : integer type
        - `"float"` : general number type
        - `"str"` : text type

    *optional :*
    - `new_column` (*str*): name of the output column.
       By default the `column` arguments is modified.

    ---

    ### Example

    **Input**

    | Column 1 |  Column 2   |  Column 3  |
    |:-------:|:--------:|:--------:|
    |  'one'  |  '2014'  |   30.0   |
    |  'two'  |  2015.0  |    '1'   |
    |   3.1   |   2016   |    450   |

    ```cson
    postprocess: [
      cast:
        column: 'Column 1'
        type: 'str'
      cast:
        column: 'Column 2'
        type: 'int'
      cast:
        column: 'Column 3'
        type: 'float'
    ]
    ```

    **Output**

    | Column 1 |  Column 2  |  Column 3  |
    |:-------:|:------:|:--------:|
    |  'one'  |  2014  |   30.0   |
    |  'two'  |  2015  |    1.0   |
    |  '3.1'  |  2016  |  450.0   |


Return a line for each bars of a waterfall chart, totals, groups, subgroups.
    Compute the variation and variation rate for each line.

    ---

    ### Parameters

    *mandatory :*
    - `date` (*str*): name of the column that id the period of each lines
    - `value` (*str*): name of the column that contains the vaue for each lines
    - `start` (*dict*):
        - `label`: text displayed under the first master column
        - `id`: value in the date col that id lines for the first period
    - `end` (*dict*):
        - `label`: text displayed under the last master column
        - `id`: value in the date col that id lines for the second period

    *optional :*
    - `upperGroup` (*dict*):
        - `id`: name of the column that contains upperGroups unique IDs
        - `label`: not required, text displayed under each upperGroups bars,
          using ID when it's absent
        - `groupsOrder`: not required, order of upperGroups
    - `insideGroup` (*dict*):
        - `id`: name of the column that contains insideGroups unique IDs
        - `label`: not required, text displayed under each insideGroups bars,
          using ID when it's absent
        - `groupsOrder`: not required, order of insideGroups
    - `filters` (*list*): columns to filters on

    ---

    ### Example

    **Input**

    | product_id   |   played | date   |   ord | category_id   | category_name   |
    |:------------:|:--------:|:------:|:-----:|:-------------:|:---------------:|
    | super clap   |       12 | t1     |     1 | clap          | Clap            |
    | clap clap    |        1 | t1     |    10 | clap          | Clap            |
    | tac          |        1 | t1     |     1 | snare         | Snare           |
    | super clap   |       10 | t2     |     1 | clap          | Clap            |
    | tac          |      100 | t2     |     1 | snare         | Snare           |
    | bom          |        1 | t2     |     1 | tom           | Tom             |

    ```cson
    waterfall:
      upperGroup:
        id: 'category_id'
        label: 'category_name'
      insideGroup:
        id: 'product_id'
        groupsOrder: 'ord'
      date: 'date'
      value: 'played'
      start:
        label: 'Trimestre 1'
        id: 't1'
      end:
        label: 'Trimester 2'
        id: 't2'
    ```

    **Output**

    |   value | label       |   variation | groups   | type   |   order |
    |:-------:|:-----------:|:-----------:|:--------:|:------:|:-------:|
    |      14 | Trimestre 1 |  NaN        | NaN      | NaN    |     NaN |
    |      -3 | Clap        |   -0.230769 | clap     | parent |     NaN |
    |      -2 | super clap  |   -0.166667 | clap     | child  |       1 |
    |      -1 | clap clap   |   -1        | clap     | child  |      10 |
    |      99 | Snare       |   99        | snare    | parent |     NaN |
    |      99 | tac         |   99        | snare    | child  |       1 |
    |       1 | Tom         |  inf        | tom      | parent |     NaN |
    |       1 | bom         |  inf        | tom      | child  |       1 |
    |     111 | Trimester 2 |  NaN        | NaN      | NaN    |     NaN |


Get the absolute numeric value of each element of a column

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the column

    *optional :*
    - `new_column` (*str*): name of the column containing the result.
      By default, no new column will be created and `column` will be replaced.

    ---

    ### Example

    **Input**

    | ENTITY | VALUE_1 | VALUE_2 |
    |:------:|:-------:|:-------:|
    | A      | -1.512  | -1.504  |
    | A      | 0.432   | 0.14    |

    ```cson
    absolute_values:
      column: 'VALUE_1'
      new_column: 'Pika'
    ```

    **Output**

    | ENTITY | VALUE_1 | VALUE_2 | Pika  |
    |:------:|:-------:|:-------:|:-----:|
    | A      | -1.512  | -1.504  | 1.512 |
    | A      | 0.432   | 0.14    | 0.432 |


Pivot the data. Reverse operation of melting

    ---

    ### Parameters

    *mandatory :*
    - `index` (*list*): names of index columns.
    - `column` (*str*): column name to pivot on
    - `value` (*str*): column name containing the value to fill the pivoted df

    *optional :*
    - `agg_function` (*str*): aggregation function to use among 'mean' (default), 'count', 'mean', 'max', 'min'

    ---

    ### Example

    **Input**

    | variable |   wave  |  year    | value |
    |:--------:|:-------:|:--------:|:-----:|
    |   toto   |  wave 1 |  2014    |  300  |
    |   toto   |  wave 1 |  2015    |  250  |
    |   toto   |  wave 1 |  2016    |  450  |

    ```cson
    pivot:
      index: ['variable','wave']
      column: 'year'
      value: 'value'
    ```

    **Output**

    | variable |   wave  |  2014  | 2015 | 2015 |
    |:--------:|:-------:|:------:|:----:|:----:|
    |   toto   |  wave 1 |  300   | 250  | 450  |