Draws the network, highlighting queues of a certain type.

        The colored vertices represent self loops of type ``edge_type``.
        Dark edges represent queues of type ``edge_type``.

        Parameters
        ----------
        edge_type : int
            The type of vertices and edges to be shown.
        **kwargs
            Any additional parameters to pass to :meth:`.draw`, and
            :meth:`.QueueNetworkDiGraph.draw_graph`

        Notes
        -----
        The colors are defined by the class attribute ``colors``. The
        relevant colors are ``vertex_active``, ``vertex_inactive``,
        ``vertex_highlight``, ``edge_active``, and ``edge_inactive``.

        Examples
        --------
        The following code highlights all edges with edge type ``2``.
        If the edge is a loop then the vertex is highlighted as well.
        In this case all edges with edge type ``2`` happen to be loops.

        >>> import queueing_tool as qt
        >>> g = qt.generate_pagerank_graph(100, seed=13)
        >>> net = qt.QueueNetwork(g, seed=13)
        >>> fname = 'edge_type_2.png'
        >>> net.show_type(2, fname=fname) # doctest: +SKIP

        .. figure:: edge_type_2-1.png
           :align: center


Simulates the network forward.

        Simulates either a specific number of events or for a specified
        amount of simulation time.

        Parameters
        ----------
        n : int (optional, default: 1)
            The number of events to simulate. If ``t`` is not given
            then this parameter is used.
        t : float (optional)
            The amount of simulation time to simulate forward. If
            given, ``t`` is used instead of ``n``.

        Raises
        ------
        QueueingToolError
            Will raise a :exc:`.QueueingToolError` if the
            ``QueueNetwork`` has not been initialized. Call
            :meth:`.initialize` before calling this method.

        Examples
        --------
        Let ``net`` denote your instance of a ``QueueNetwork``. Before
        you simulate, you need to initialize the network, which allows
        arrivals from outside the network. To initialize with 2 (random
        chosen) edges accepting arrivals run:

        >>> import queueing_tool as qt
        >>> g = qt.generate_pagerank_graph(100, seed=50)
        >>> net = qt.QueueNetwork(g, seed=50)
        >>> net.initialize(2)

        To simulate the network 50000 events run:

        >>> net.num_events
        0
        >>> net.simulate(50000)
        >>> net.num_events
        50000

        To simulate the network for at least 75 simulation time units
        run:

        >>> t0 = net.current_time
        >>> net.simulate(t=75)
        >>> t1 = net.current_time
        >>> t1 - t0 # doctest: +ELLIPSIS
        75...


Tells the queues to collect data on agents' arrival, service
        start, and departure times.

        If none of the parameters are given then every
        :class:`.QueueServer` will start collecting data.

        Parameters
        ----------
        queues : :any:`int`, *array_like* (optional)
            The edge index (or an iterable of edge indices) identifying
            the :class:`QueueServer(s)<.QueueServer>` that will start
            collecting data.
        edge : 2-tuple of int or *array_like* (optional)
            Explicitly specify which queues will collect data. Must be
            either:

            * A 2-tuple of the edge's source and target vertex
              indices, or
            * An iterable of 2-tuples of the edge's source and
              target vertex indices.

        edge_type : int or an iterable of int (optional)
            A integer, or a collection of integers identifying which
            edge types will be set active.


Tells the queues to stop collecting data on agents.

        If none of the parameters are given then every
        :class:`.QueueServer` will stop collecting data.

        Parameters
        ----------
        queues : int, *array_like* (optional)
            The edge index (or an iterable of edge indices) identifying
            the :class:`QueueServer(s)<.QueueServer>` that will stop
            collecting data.
        edge : 2-tuple of int or *array_like* (optional)
            Explicitly specify which queues will stop collecting data.
            Must be either:

            * A 2-tuple of the edge's source and target vertex
              indices, or
            * An iterable of 2-tuples of the edge's source and
              target vertex indices.

        edge_type : int or an iterable of int (optional)
            A integer, or a collection of integers identifying which
            edge types will stop collecting data.


Returns the routing probabilities for each vertex in the
        graph.

        Parameters
        ----------
        return_matrix : bool (optional, the default is ``True``)
            Specifies whether an :class:`~numpy.ndarray` is returned.
            If ``False``, a dict is returned instead.

        Returns
        -------
        out : a dict or :class:`~numpy.ndarray`
            The transition probabilities for each vertex in the graph.
            If ``out`` is an :class:`~numpy.ndarray`, then
            ``out[v, u]`` returns the probability of a transition from
            vertex ``v`` to vertex ``u``. If ``out`` is a dict
            then ``out_edge[v][u]`` is the probability of moving from
            vertex ``v`` to the vertex ``u``.

        Examples
        --------
        Lets change the routing probabilities:

        >>> import queueing_tool as qt
        >>> import networkx as nx
        >>> g = nx.sedgewick_maze_graph()
        >>> net = qt.QueueNetwork(g)

        Below is an adjacency list for the graph ``g``.

        >>> ans = qt.graph2dict(g, False)
        >>> {k: sorted(v) for k, v in ans.items()}
        ...                         # doctest: +NORMALIZE_WHITESPACE
        {0: [2, 5, 7],
         1: [7],
         2: [0, 6],
         3: [4, 5],
         4: [3, 5, 6, 7],
         5: [0, 3, 4],
         6: [2, 4],
         7: [0, 1, 4]}

        The default transition matrix is every out edge being equally
        likely:

        >>> net.transitions(False)  # doctest: +ELLIPSIS
        ...                         # doctest: +NORMALIZE_WHITESPACE
        {0: {2: 0.333..., 5: 0.333..., 7: 0.333...},
         1: {7: 1.0},
         2: {0: 0.5, 6: 0.5},
         3: {4: 0.5, 5: 0.5},
         4: {3: 0.25, 5: 0.25, 6: 0.25, 7: 0.25},
         5: {0: 0.333..., 3: 0.333..., 4: 0.333...},
         6: {2: 0.5, 4: 0.5},
         7: {0: 0.333..., 1: 0.333..., 4: 0.333...}}

        Now we will generate a random routing matrix:

        >>> mat = qt.generate_transition_matrix(g, seed=96)
        >>> net.set_transitions(mat)
        >>> net.transitions(False)  # doctest: +ELLIPSIS
        ...                         # doctest: +NORMALIZE_WHITESPACE
        {0: {2: 0.112..., 5: 0.466..., 7: 0.420...},
         1: {7: 1.0},
         2: {0: 0.561..., 6: 0.438...},
         3: {4: 0.545..., 5: 0.454...},
         4: {3: 0.374..., 5: 0.381..., 6: 0.026..., 7: 0.217...},
         5: {0: 0.265..., 3: 0.460..., 4: 0.274...},
         6: {2: 0.673..., 4: 0.326...},
         7: {0: 0.033..., 1: 0.336..., 4: 0.630...}}

        What this shows is the following: when an :class:`.Agent` is at
        vertex ``2`` they will transition to vertex ``0`` with
        probability ``0.561`` and route to vertex ``6`` probability
        ``0.438``, when at vertex ``6`` they will transition back to
        vertex ``2`` with probability ``0.673`` and route vertex ``4``
        probability ``0.326``, etc.


Returns the number of elements in the set that ``s`` belongs to.

        Parameters
        ----------
        s : object
            An object

        Returns
        -------
        out : int
            The number of elements in the set that ``s`` belongs to.


Locates the leader of the set to which the element ``s`` belongs.

        Parameters
        ----------
        s : object
            An object that the ``UnionFind`` contains.

        Returns
        -------
        object
            The leader of the set that contains ``s``.


Merges the set that contains ``a`` with the set that contains ``b``.

        Parameters
        ----------
        a, b : objects
            Two objects whose sets are to be merged.


Generates a random transition matrix for the graph ``g``.

    Parameters
    ----------
    g : :any:`networkx.DiGraph`, :class:`numpy.ndarray`, dict, etc.
        Any object that :any:`DiGraph<networkx.DiGraph>` accepts.
    seed : int (optional)
        An integer used to initialize numpy's psuedo-random number
        generator.

    Returns
    -------
    mat : :class:`~numpy.ndarray`
        Returns a transition matrix where ``mat[i, j]`` is the
        probability of transitioning from vertex ``i`` to vertex ``j``.
        If there is no edge connecting vertex ``i`` to vertex ``j``
        then ``mat[i, j] = 0``.


Creates a random graph where the vertex types are
    selected using their pagerank.

    Calls :func:`.minimal_random_graph` and then
    :func:`.set_types_rank` where the ``rank`` keyword argument
    is given by :func:`networkx.pagerank`.

    Parameters
    ----------
    num_vertices : int (optional, the default is 250)
        The number of vertices in the graph.
    **kwargs :
        Any parameters to send to :func:`.minimal_random_graph` or
        :func:`.set_types_rank`.

    Returns
    -------
    :class:`.QueueNetworkDiGraph`
        A graph with a ``pos`` vertex property and the ``edge_type``
        edge property.

    Notes
    -----
    This function sets the edge types of a graph to be either 1, 2, or
    3. It sets the vertices to type 2 by selecting the top
    ``pType2 * g.number_of_nodes()`` vertices given by the
    :func:`~networkx.pagerank` of the graph. A loop is added
    to all vertices identified this way (if one does not exist
    already). It then randomly sets vertices close to the type 2
    vertices as type 3, and adds loops to these vertices as well. These
    loops then have edge types that correspond to the vertices type.
    The rest of the edges are set to type 1.


Yield all of the documentation for trait definitions on a class object.


Add lines to the block.


We are transitioning from a noncomment to a comment.


Possibly add a new comment.

        Only adds a new comment if this comment is the only thing on the line.
        Otherwise, it extends the noncomment block.


Make the index mapping lines of actual code to their associated
        prefix comments.


Read complete DSMR telegram's from the serial interface and parse it
        into CosemObject's and MbusObject's

        :rtype: generator


Creates a DSMR asyncio protocol.


Creates a DSMR asyncio protocol coroutine using serial port.


Add incoming data to buffer.


Send off parsed telegram to handling callback.


Parse telegram from string to dict.

        The telegram str type makes python 2.x integration easier.

        :param str telegram_data: full telegram from start ('/') to checksum
            ('!ABCD') including line endings in between the telegram's lines
        :rtype: dict
        :returns: Shortened example:
            {
                ..
                r'\d-\d:96\.1\.1.+?\r\n': <CosemObject>,  # EQUIPMENT_IDENTIFIER
                r'\d-\d:1\.8\.1.+?\r\n': <CosemObject>,   # ELECTRICITY_USED_TARIFF_1
                r'\d-\d:24\.3\.0.+?\r\n.+?\r\n': <MBusObject>,  # GAS_METER_READING
                ..
            }
        :raises ParseError:
        :raises InvalidChecksumError:


Loads config from string or dict


Converts the configuration dictionary into the corresponding configuration format

        :param files: whether to include "additional files" in the output or not;
                      defaults to ``True``
        :returns: string with output


Returns a ``BytesIO`` instance representing an in-memory tar.gz archive
        containing the native router configuration.

        :returns: in-memory tar.gz archive, instance of ``BytesIO``


Adds a single file in tarfile instance.

        :param tar: tarfile instance
        :param name: string representing filename or path
        :param contents: string representing file contents
        :param mode: string representing file mode, defaults to 644
        :returns: None


Parses a native configuration and converts
        it to a NetJSON configuration dictionary


Merges ``list2`` on top of ``list1``.

    If both lists contain dictionaries which have keys specified
    in ``identifiers`` which have equal values, those dicts will
    be merged (dicts in ``list2`` will override dicts in ``list1``).
    The remaining elements will be summed in order to create a list
    which contains elements of both lists.

    :param list1: ``list`` from template
    :param list2: ``list`` from config
    :param identifiers: ``list`` or ``None``
    :returns: merged ``list``


Evaluates variables in ``data``

    :param data: data structure containing variables, may be
                 ``str``, ``dict`` or ``list``
    :param context: ``dict`` containing variables
    :returns: modified data structure


Looks for a key in a dictionary, if found returns
        a deepcopied value, otherwise returns default value


Loops over item and performs type casting
        according to supplied schema fragment


generates install.sh and adds it to included files


generates tc_script.sh and adds it to included files


Renders configuration by using the jinja2 templating engine


converts NetJSON address to
        UCI intermediate data structure


converts NetJSON interface to
        UCI intermediate data structure


deletes NetJSON address keys


converts NetJSON bridge to
        UCI intermediate data structure


determines UCI interface "proto" option


determines UCI interface "dns" option


only for mac80211 driver


determines NetJSON protocol radio attribute


Returns a configuration dictionary representing an OpenVPN client configuration
        that is compatible with the passed server configuration.

        :param host: remote VPN server
        :param server: dictionary representing a single OpenVPN server configuration
        :param ca_path: optional string representing path to CA, will consequently add
                        a file in the resulting configuration dictionary
        :param ca_contents: optional string representing contents of CA file
        :param cert_path: optional string representing path to certificate, will consequently add
                        a file in the resulting configuration dictionary
        :param cert_contents: optional string representing contents of cert file
        :param key_path: optional string representing path to key, will consequently add
                        a file in the resulting configuration dictionary
        :param key_contents: optional string representing contents of key file
        :returns: dictionary representing a single OpenVPN client configuration


returns a list of NetJSON extra files for automatically generated clients
        produces side effects in ``client`` dictionary


parse requirements.txt, ignore links, exclude comments


Get all facts of this node. Additional arguments may also be
        specified that will be passed to the query function.


Get a single fact from this node.


Get all resources of this node or all resources of the specified
        type. Additional arguments may also be specified that will be passed
        to the query function.


Get all reports for this node. Additional arguments may also be
        specified that will be passed to the query function.


A base_url that will be used to construct the final
        URL we're going to query against.

        :returns: A URL of the form: ``proto://host:port``.
        :rtype: :obj:`string`


The complete URL we will end up querying. Depending on the
        endpoint we pass in  this will result in different URL's with
        different prefixes.

        :param endpoint: The PuppetDB API endpoint we want to query.
        :type endpoint: :obj:`string`
        :param path: An additional path if we don't wish to query the\
                bare endpoint.
        :type path: :obj:`string`

        :returns: A URL constructed from :func:`base_url` with the\
                apropraite API version/prefix and the rest of the path added\
                to it.
        :rtype: :obj:`string`


Query for nodes by either name or query. If both aren't
        provided this will return a list of all nodes. This method
        also fetches the nodes status and event counts of the latest
        report from puppetdb.

        :param with_status: (optional) include the node status in the\
                           returned nodes
        :type with_status: :bool:
        :param unreported: (optional) amount of hours when a node gets
                           marked as unreported
        :type unreported: :obj:`None` or integer
        :param \*\*kwargs: The rest of the keyword arguments are passed
                           to the _query function

        :returns: A generator yieling Nodes.
        :rtype: :class:`pypuppetdb.types.Node`


Gets a single node from PuppetDB.

        :param name: The name of the node search.
        :type name: :obj:`string`

        :return: An instance of Node
        :rtype: :class:`pypuppetdb.types.Node`


Get the available catalog for a given node.

        :param node: (Required) The name of the PuppetDB node.
        :type: :obj:`string`

        :returns: An instance of Catalog
        :rtype: :class:`pypuppetdb.types.Catalog`


Connect with PuppetDB. This will return an object allowing you
    to query the API through its methods.

    :param host: (Default: 'localhost;) Hostname or IP of PuppetDB.
    :type host: :obj:`string`

    :param port: (Default: '8080') Port on which to talk to PuppetDB.
    :type port: :obj:`int`

    :param ssl_verify: (optional) Verify PuppetDB server certificate.
    :type ssl_verify: :obj:`bool` or :obj:`string` True, False or filesystem \
            path to CA certificate.

    :param ssl_key: (optional) Path to our client secret key.
    :type ssl_key: :obj:`None` or :obj:`string` representing a filesystem\
            path.

    :param ssl_cert: (optional) Path to our client certificate.
    :type ssl_cert: :obj:`None` or :obj:`string` representing a filesystem\
            path.

    :param timeout: (Default: 10) Number of seconds to wait for a response.
    :type timeout: :obj:`int`

    :param protocol: (optional) Explicitly specify the protocol to be used
            (especially handy when using HTTPS with ssl_verify=False and
            without certs)
    :type protocol: :obj:`None` or :obj:`string`

    :param url_path: (Default: '/') The URL path where PuppetDB is served
    :type url_path: :obj:`None` or :obj:`string`

    :param username: (optional) The username to use for HTTP basic
            authentication
    :type username: :obj:`None` or :obj:`string`

    :param password: (optional) The password to use for HTTP basic
            authentication
    :type password: :obj:`None` or :obj:`string`

    :param token: (optional) The x-auth token to use for X-Authentication
    :type token: :obj:`None` or :obj:`string`


The Master has been started from the command line. Execute ad-hoc tests if desired.


Direct operate a set of commands

        :param command_set: set of command headers
        :param callback: callback that will be invoked upon completion or failure
        :param config: optional configuration that controls normal callbacks and allows the user to be specified for SA


Select and operate a single command

        :param command: command to operate
        :param index: index of the command
        :param callback: callback that will be invoked upon completion or failure
        :param config: optional configuration that controls normal callbacks and allows the user to be specified for SA


Select and operate a set of commands

        :param command_set: set of command headers
        :param callback: callback that will be invoked upon completion or failure
        :param config: optional configuration that controls normal callbacks and allows the user to be specified for SA


The Outstation has been started from the command line. Execute ad-hoc tests if desired.


The Master sent an Operate command to the Outstation. Handle it.

        :param command: ControlRelayOutputBlock,
                        AnalogOutputInt16, AnalogOutputInt32, AnalogOutputFloat32, or AnalogOutputDouble64.
        :param index: int
        :param op_type: OperateType
        :return: CommandStatus


Create Bloomberg connection

    Returns:
        (Bloomberg connection, if connection is new)


Stop and destroy Bloomberg connection


Parse markdown as description


Standardized earning outputs and add percentage by each blocks

    Args:
        data: earning data block
        header: earning headers

    Returns:
        pd.DataFrame

    Examples:
        >>> format_earning(
        ...     data=pd.read_pickle('xbbg/tests/data/sample_earning.pkl'),
        ...     header=pd.read_pickle('xbbg/tests/data/sample_earning_header.pkl')
        ... ).round(2)
                         level  fy2017  fy2017_pct
        Asia-Pacific       1.0  3540.0       66.43
           China           2.0  1747.0       49.35
           Japan           2.0  1242.0       35.08
           Singapore       2.0   551.0       15.56
        United States      1.0  1364.0       25.60
        Europe             1.0   263.0        4.94
        Other Countries    1.0   162.0        3.04


Format `pdblp` outputs to column-based results

    Args:
        data: `pdblp` result
        source: `bdp` or `bds`
        col_maps: rename columns with these mappings

    Returns:
        pd.DataFrame

    Examples:
        >>> format_output(
        ...     data=pd.read_pickle('xbbg/tests/data/sample_bdp.pkl'),
        ...     source='bdp'
        ... ).reset_index()
                  ticker                        name
        0  QQQ US Equity  INVESCO QQQ TRUST SERIES 1
        1  SPY US Equity      SPDR S&P 500 ETF TRUST
        >>> format_output(
        ...     data=pd.read_pickle('xbbg/tests/data/sample_dvd.pkl'),
        ...     source='bds', col_maps={'Dividend Frequency': 'dvd_freq'}
        ... ).loc[:, ['ex_date', 'dividend_amount', 'dvd_freq']].reset_index()
                ticker     ex_date  dividend_amount dvd_freq
        0  C US Equity  2018-02-02             0.32  Quarter


Format intraday data

    Args:
        data: pd.DataFrame from bdib
        ticker: ticker

    Returns:
        pd.DataFrame

    Examples:
        >>> format_intraday(
        ...     data=pd.read_parquet('xbbg/tests/data/sample_bdib.parq'),
        ...     ticker='SPY US Equity',
        ... ).xs('close', axis=1, level=1, drop_level=False)
        ticker                    SPY US Equity
        field                             close
        2018-12-28 09:30:00-05:00        249.67
        2018-12-28 09:31:00-05:00        249.54
        2018-12-28 09:32:00-05:00        249.22
        2018-12-28 09:33:00-05:00        249.01
        2018-12-28 09:34:00-05:00        248.86
        >>> format_intraday(
        ...     data=pd.read_parquet('xbbg/tests/data/sample_bdib.parq'),
        ...     ticker='SPY US Equity', price_only=True
        ... )
        ticker                     SPY US Equity
        2018-12-28 09:30:00-05:00         249.67
        2018-12-28 09:31:00-05:00         249.54
        2018-12-28 09:32:00-05:00         249.22
        2018-12-28 09:33:00-05:00         249.01
        2018-12-28 09:34:00-05:00         248.86


Logging info for given tickers and fields

    Args:
        tickers: tickers
        flds: fields

    Returns:
        str

    Examples:
        >>> print(info_qry(
        ...     tickers=['NVDA US Equity'], flds=['Name', 'Security_Name']
        ... ))
        tickers: ['NVDA US Equity']
        fields:  ['Name', 'Security_Name']


Bloomberg historical data

    Args:
        tickers: ticker(s)
        flds: field(s)
        start_date: start date
        end_date: end date - default today
        adjust: `all`, `dvd`, `normal`, `abn` (=abnormal), `split`, `-` or None
                exact match of above words will adjust for corresponding events
                Case 0: `-` no adjustment for dividend or split
                Case 1: `dvd` or `normal|abn` will adjust for all dividends except splits
                Case 2: `adjust` will adjust for splits and ignore all dividends
                Case 3: `all` == `dvd|split` == adjust for all
                Case 4: None == Bloomberg default OR use kwargs
        **kwargs: overrides

    Returns:
        pd.DataFrame

    Examples:
        >>> res = bdh(
        ...     tickers='VIX Index', flds=['High', 'Low', 'Last_Price'],
        ...     start_date='2018-02-05', end_date='2018-02-07',
        ... ).round(2).transpose()
        >>> res.index.name = None
        >>> res.columns.name = None
        >>> res
                              2018-02-05  2018-02-06  2018-02-07
        VIX Index High             38.80       50.30       31.64
                  Low              16.80       22.42       21.17
                  Last_Price       37.32       29.98       27.73
        >>> bdh(
        ...     tickers='AAPL US Equity', flds='Px_Last',
        ...     start_date='20140605', end_date='20140610', adjust='-'
        ... ).round(2)
        ticker     AAPL US Equity
        field             Px_Last
        2014-06-05         647.35
        2014-06-06         645.57
        2014-06-09          93.70
        2014-06-10          94.25
        >>> bdh(
        ...     tickers='AAPL US Equity', flds='Px_Last',
        ...     start_date='20140606', end_date='20140609',
        ...     CshAdjNormal=False, CshAdjAbnormal=False, CapChg=False,
        ... ).round(2)
        ticker     AAPL US Equity
        field             Px_Last
        2014-06-06         645.57
        2014-06-09          93.70


Bloomberg intraday bar data within market session

    Args:
        ticker: ticker
        dt: date
        session: examples include
                 day_open_30, am_normal_30_30, day_close_30, allday_exact_0930_1000
        **kwargs:
            ref: reference ticker or exchange for timezone
            keep_tz: if keep tz if reference ticker / exchange is given
            start_time: start time
            end_time: end time
            typ: [TRADE, BID, ASK, BID_BEST, ASK_BEST, BEST_BID, BEST_ASK]

    Returns:
        pd.DataFrame


Earning exposures by Geo or Products

    Args:
        ticker: ticker name
        by: [G(eo), P(roduct)]
        typ: type of earning, start with `PG_` in Bloomberg FLDS - default `Revenue`
        ccy: currency of earnings
        level: hierarchy level of earnings

    Returns:
        pd.DataFrame

    Examples:
        >>> data = earning('AMD US Equity', Eqy_Fund_Year=2017, Number_Of_Periods=1)
        >>> data.round(2)
                         level  fy2017  fy2017_pct
        Asia-Pacific       1.0  3540.0       66.43
           China           2.0  1747.0       49.35
           Japan           2.0  1242.0       35.08
           Singapore       2.0   551.0       15.56
        United States      1.0  1364.0       25.60
        Europe             1.0   263.0        4.94
        Other Countries    1.0   162.0        3.04


Active futures contract

    Args:
        ticker: futures ticker, i.e., ESA Index, Z A Index, CLA Comdty, etc.
        dt: date

    Returns:
        str: ticker name


Get proper ticker from generic ticker

    Args:
        gen_ticker: generic ticker
        dt: date
        freq: futures contract frequency
        log: level of logs

    Returns:
        str: exact futures ticker


Check exchange hours vs local hours

    Args:
        tickers: list of tickers
        tz_exch: exchange timezone
        tz_loc: local timezone

    Returns:
        Local and exchange hours


Data file location for Bloomberg historical data

    Args:
        ticker: ticker name
        dt: date
        typ: [TRADE, BID, ASK, BID_BEST, ASK_BEST, BEST_BID, BEST_ASK]

    Returns:
        file location

    Examples:
        >>> os.environ['BBG_ROOT'] = ''
        >>> hist_file(ticker='ES1 Index', dt='2018-08-01') == ''
        True
        >>> os.environ['BBG_ROOT'] = '/data/bbg'
        >>> hist_file(ticker='ES1 Index', dt='2018-08-01')
        '/data/bbg/Index/ES1 Index/TRADE/2018-08-01.parq'


Data file location for Bloomberg reference data

    Args:
        ticker: ticker name
        fld: field
        has_date: whether add current date to data file
        cache: if has_date is True, whether to load file from latest cached
        ext: file extension
        **kwargs: other overrides passed to ref function

    Returns:
        file location

    Examples:
        >>> import shutil
        >>>
        >>> os.environ['BBG_ROOT'] = ''
        >>> ref_file('BLT LN Equity', fld='Crncy') == ''
        True
        >>> os.environ['BBG_ROOT'] = '/data/bbg'
        >>> ref_file('BLT LN Equity', fld='Crncy', cache=True)
        '/data/bbg/Equity/BLT LN Equity/Crncy/ovrd=None.parq'
        >>> ref_file('BLT LN Equity', fld='Crncy')
        ''
        >>> cur_dt = utils.cur_time(tz=utils.DEFAULT_TZ)
        >>> ref_file(
        ...     'BLT LN Equity', fld='DVD_Hist_All', has_date=True, cache=True,
        ... ).replace(cur_dt, '[cur_date]')
        '/data/bbg/Equity/BLT LN Equity/DVD_Hist_All/asof=[cur_date], ovrd=None.parq'
        >>> ref_file(
        ...     'BLT LN Equity', fld='DVD_Hist_All', has_date=True,
        ...     cache=True, DVD_Start_Dt='20180101',
        ... ).replace(cur_dt, '[cur_date]')[:-5]
        '/data/bbg/Equity/BLT LN Equity/DVD_Hist_All/asof=[cur_date], DVD_Start_Dt=20180101'
        >>> sample = 'asof=2018-11-02, DVD_Start_Dt=20180101, DVD_End_Dt=20180501.pkl'
        >>> root_path = 'xbbg/tests/data'
        >>> sub_path = f'{root_path}/Equity/AAPL US Equity/DVD_Hist_All'
        >>> os.environ['BBG_ROOT'] = root_path
        >>> for tmp_file in files.all_files(sub_path): os.remove(tmp_file)
        >>> files.create_folder(sub_path)
        >>> sample in shutil.copy(f'{root_path}/{sample}', sub_path)
        True
        >>> new_file = ref_file(
        ...     'AAPL US Equity', 'DVD_Hist_All', DVD_Start_Dt='20180101',
        ...     has_date=True, cache=True, ext='pkl'
        ... )
        >>> new_file.split('/')[-1] == f'asof={cur_dt}, DVD_Start_Dt=20180101.pkl'
        True
        >>> old_file = 'asof=2018-11-02, DVD_Start_Dt=20180101, DVD_End_Dt=20180501.pkl'
        >>> old_full = '/'.join(new_file.split('/')[:-1] + [old_file])
        >>> updated_file = old_full.replace('2018-11-02', cur_dt)
        >>> updated_file in shutil.copy(old_full, updated_file)
        True
        >>> exist_file = ref_file(
        ...     'AAPL US Equity', 'DVD_Hist_All', DVD_Start_Dt='20180101',
        ...     has_date=True, cache=True, ext='pkl'
        ... )
        >>> exist_file == updated_file
        False
        >>> exist_file = ref_file(
        ...     'AAPL US Equity', 'DVD_Hist_All', DVD_Start_Dt='20180101',
        ...     DVD_End_Dt='20180501', has_date=True, cache=True, ext='pkl'
        ... )
        >>> exist_file == updated_file
        True


Check whether data is done for the day and save

    Args:
        data: data
        ticker: ticker
        dt: date
        typ: [TRADE, BID, ASK, BID_BEST, ASK_BEST, BEST_BID, BEST_ASK]

    Examples:
        >>> os.environ['BBG_ROOT'] = 'xbbg/tests/data'
        >>> sample = pd.read_parquet('xbbg/tests/data/aapl.parq')
        >>> save_intraday(sample, 'AAPL US Equity', '2018-11-02')
        >>> # Invalid exchange
        >>> save_intraday(sample, 'AAPL XX Equity', '2018-11-02')
        >>> # Invalid empty data
        >>> save_intraday(pd.DataFrame(), 'AAPL US Equity', '2018-11-02')
        >>> # Invalid date - too close
        >>> cur_dt = utils.cur_time()
        >>> save_intraday(sample, 'AAPL US Equity', cur_dt)


Exchange info for given ticker

    Args:
        ticker: ticker or exchange

    Returns:
        pd.Series

    Examples:
        >>> exch_info('SPY US Equity')
        tz        America/New_York
        allday      [04:00, 20:00]
        day         [09:30, 16:00]
        pre         [04:00, 09:30]
        post        [16:01, 20:00]
        dtype: object
        >>> exch_info('ES1 Index')
        tz        America/New_York
        allday      [18:00, 17:00]
        day         [08:00, 17:00]
        dtype: object
        >>> exch_info('Z 1 Index')
        tz         Europe/London
        allday    [01:00, 21:00]
        day       [01:00, 21:00]
        dtype: object
        >>> exch_info('TESTTICKER Corp').empty
        True
        >>> exch_info('US')
        tz        America/New_York
        allday      [04:00, 20:00]
        day         [09:30, 16:00]
        pre         [04:00, 09:30]
        post        [16:01, 20:00]
        dtype: object


Get info for given market

    Args:
        ticker: Bloomberg full ticker

    Returns:
        dict

    Examples:
        >>> info = market_info('SHCOMP Index')
        >>> info['exch']
        'EquityChina'
        >>> info = market_info('ICICIC=1 IS Equity')
        >>> info['freq'], info['is_fut']
        ('M', True)
        >>> info = market_info('INT1 Curncy')
        >>> info['freq'], info['is_fut']
        ('M', True)
        >>> info = market_info('CL1 Comdty')
        >>> info['freq'], info['is_fut']
        ('M', True)
        >>> # Wrong tickers
        >>> market_info('C XX Equity')
        {}
        >>> market_info('XXX Comdty')
        {}
        >>> market_info('Bond_ISIN Corp')
        {}
        >>> market_info('XYZ Index')
        {}
        >>> market_info('XYZ Curncy')
        {}


Currency pair info

    Args:
        local: local currency
        base: base currency

    Returns:
        CurrencyPair

    Examples:
        >>> ccy_pair(local='HKD', base='USD')
        CurrencyPair(ticker='HKD Curncy', factor=1.0, power=1)
        >>> ccy_pair(local='GBp')
        CurrencyPair(ticker='GBP Curncy', factor=100, power=-1)
        >>> ccy_pair(local='USD', base='GBp')
        CurrencyPair(ticker='GBP Curncy', factor=0.01, power=1)
        >>> ccy_pair(local='XYZ', base='USD')
        CurrencyPair(ticker='', factor=1.0, power=1)
        >>> ccy_pair(local='GBP', base='GBp')
        CurrencyPair(ticker='', factor=0.01, power=1)
        >>> ccy_pair(local='GBp', base='GBP')
        CurrencyPair(ticker='', factor=100.0, power=1)


Market close time for ticker

    Args:
        ticker: ticker name
        dt: date
        timing: [EOD (default), BOD]
        tz: conversion to timezone

    Returns:
        str: date & time

    Examples:
        >>> market_timing('7267 JT Equity', dt='2018-09-10')
        '2018-09-10 14:58'
        >>> market_timing('7267 JT Equity', dt='2018-09-10', tz=timezone.TimeZone.NY)
        '2018-09-10 01:58:00-04:00'
        >>> market_timing('7267 JT Equity', dt='2018-01-10', tz='NY')
        '2018-01-10 00:58:00-05:00'
        >>> market_timing('7267 JT Equity', dt='2018-09-10', tz='SPX Index')
        '2018-09-10 01:58:00-04:00'
        >>> market_timing('8035 JT Equity', dt='2018-09-10', timing='BOD')
        '2018-09-10 09:01'
        >>> market_timing('Z 1 Index', dt='2018-09-10', timing='FINISHED')
        '2018-09-10 21:00'
        >>> market_timing('TESTTICKER Corp', dt='2018-09-10')
        ''


Load parameters for assets

    Args:
        cat: category

    Returns:
        dict

    Examples:
        >>> import pandas as pd
        >>>
        >>> assets = load_info(cat='assets')
        >>> all(cat in assets for cat in ['Equity', 'Index', 'Curncy', 'Corp'])
        True
        >>> os.environ['BBG_PATH'] = ''
        >>> exch = load_info(cat='exch')
        >>> pd.Series(exch['EquityUS']).allday
        [400, 2000]
        >>> test_root = f'{PKG_PATH}/tests'
        >>> os.environ['BBG_PATH'] = test_root
        >>> ovrd_exch = load_info(cat='exch')
        >>> # Somehow os.environ is not set properly in doctest environment
        >>> ovrd_exch.update(_load_yaml_(f'{test_root}/markets/exch.yml'))
        >>> pd.Series(ovrd_exch['EquityUS']).allday
        [300, 2100]


Convert YAML input to hours

    Args:
        num: number in YMAL file, e.g., 900, 1700, etc.

    Returns:
        str

    Examples:
        >>> to_hour(900)
        '09:00'
        >>> to_hour(1700)
        '17:00'


Make folder as well as all parent folders if not exists

    Args:
        path_name: full path name
        is_file: whether input is name of file


Search all files with criteria
    Returned list will be sorted by last modified

    Args:
        path_name: full path name
        keyword: keyword to search
        ext: file extensions, split by ','
        full_path: whether return full path (default True)
        has_date: whether has date in file name (default False)
        date_fmt: date format to check for has_date parameter

    Returns:
        list: all file names with criteria fulfilled


Search all folders with criteria
    Returned list will be sorted by last modified

    Args:
        path_name: full path name
        keyword: keyword to search
        has_date: whether has date in file name (default False)
        date_fmt: date format to check for has_date parameter

    Returns:
        list: all folder names fulfilled criteria


Sort files or folders by modified time

    Args:
        files_or_folders: list of files or folders

    Returns:
        list


Filter files or dates by date patterns

    Args:
        files_or_folders: list of files or folders
        date_fmt: date format

    Returns:
        list


File modified time in python

    Args:
        file_name: file name

    Returns:
        pd.Timestamp


Get interval from defined session

    Args:
        ticker: ticker
        session: session

    Returns:
        Session of start_time and end_time

    Examples:
        >>> get_interval('005490 KS Equity', 'day_open_30')
        Session(start_time='09:00', end_time='09:30')
        >>> get_interval('005490 KS Equity', 'day_normal_30_20')
        Session(start_time='09:31', end_time='15:00')
        >>> get_interval('005490 KS Equity', 'day_close_20')
        Session(start_time='15:01', end_time='15:20')
        >>> get_interval('700 HK Equity', 'am_open_30')
        Session(start_time='09:30', end_time='10:00')
        >>> get_interval('700 HK Equity', 'am_normal_30_30')
        Session(start_time='10:01', end_time='11:30')
        >>> get_interval('700 HK Equity', 'am_close_30')
        Session(start_time='11:31', end_time='12:00')
        >>> get_interval('ES1 Index', 'day_exact_2130_2230')
        Session(start_time=None, end_time=None)
        >>> get_interval('ES1 Index', 'allday_exact_2130_2230')
        Session(start_time='21:30', end_time='22:30')
        >>> get_interval('ES1 Index', 'allday_exact_2130_0230')
        Session(start_time='21:30', end_time='02:30')
        >>> get_interval('AMLP US', 'day_open_30')
        Session(start_time=None, end_time=None)
        >>> get_interval('7974 JP Equity', 'day_normal_180_300') is SessNA
        True
        >>> get_interval('Z 1 Index', 'allday_normal_30_30')
        Session(start_time='01:31', end_time='20:30')
        >>> get_interval('GBP Curncy', 'day')
        Session(start_time='17:02', end_time='17:00')


Shift start time by mins

    Args:
        start_time: start time in terms of HH:MM string
        mins: number of minutes (+ / -)

    Returns:
        end time in terms of HH:MM string


Time intervals for market open

        Args:
            session: [allday, day, am, pm, night]
            mins: mintues after open

        Returns:
            Session of start_time and end_time


Time intervals for market close

        Args:
            session: [allday, day, am, pm, night]
            mins: mintues before close

        Returns:
            Session of start_time and end_time


Explicitly specify start time and end time

        Args:
            session: predefined session
            start_time: start time in terms of HHMM string
            end_time: end time in terms of HHMM string

        Returns:
            Session of start_time and end_time


Convert to tz

    Args:
        dt: date time
        to_tz: to tz
        from_tz: from tz - will be ignored if tz from dt is given

    Returns:
        str: date & time

    Examples:
        >>> dt_1 = pd.Timestamp('2018-09-10 16:00', tz='Asia/Hong_Kong')
        >>> tz_convert(dt_1, to_tz='NY')
        '2018-09-10 04:00:00-04:00'
        >>> dt_2 = pd.Timestamp('2018-01-10 16:00')
        >>> tz_convert(dt_2, to_tz='HK', from_tz='NY')
        '2018-01-11 05:00:00+08:00'
        >>> dt_3 = '2018-09-10 15:00'
        >>> tz_convert(dt_3, to_tz='NY', from_tz='JP')
        '2018-09-10 02:00:00-04:00'


Full infomation for missing query


Check number of trials for missing values

    Returns:
        int: number of trials already tried


Decorator for public views that do not require authentication
    Sets an attribute in the fuction STRONGHOLD_IS_PUBLIC to True


Get the version of the package from the given file by
    executing it and extracting the given `name`.


Find all of the packages.


Echo a command before running it.  Defaults to repo as cwd


Return a Command that checks that certain files exist.

    Raises a ValueError if any of the files are missing.

    Note: The check is skipped if the `--skip-npm` flag is used.


Wrap a setup command

    Parameters
    ----------
    cmds: list(str)
        The names of the other commands to run prior to the command.
    strict: boolean, optional
        Whether to raise errors when a pre-command fails.


Expand data file specs into valid data files metadata.

    Parameters
    ----------
    data_specs: list of tuples
        See [createcmdclass] for description.
    existing: list of tuples
        The existing distribution data_files metadata.

    Returns
    -------
    A valid list of data_files items.


Translate and compile a glob pattern to a regular expression matcher.


Iterate over all the parts of a path.

    Splits path recursively with os.path.split().


Join translated glob pattern parts.

    This is different from a simple join, as care need to be taken
    to allow ** to match ZERO or more directories.


Translate a glob PATTERN PART to a regular expression.


Send DDL to create the specified `table`

        :Parameters:
          - `table`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader.Table` object that represents the table to read/write.

        Returns None


Send DDL to create the specified `table` indexes

        :Parameters:
          - `table`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader.Table` object that represents the table to read/write.

        Returns None


Send DDL to create the specified `table` triggers

        :Parameters:
          - `table`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader.Table` object that represents the table to read/write.

        Returns None


Write the contents of `table`

        :Parameters:
          - `table`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader.Table` object that represents the table to read/write.
          - `reader`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader` object that allows reading from the data source.

        Returns None


Write TRIGGERs existing on `table` to the output file

        :Parameters:
          - `table`: an instance of a :py:class:`mysql2pgsql.lib.mysql_reader.MysqlReader.Table` object that represents the table to read/write.

        Returns None


Utility for sending a predefined request and printing response as well
    as storing messages in a list, useful for testing

    Parameters
    ----------
    session: blpapi.session.Session
    request: blpapi.request.Request
        Request to be sent

    Returns
    -------
        List of all messages received


Initialize blpapi.Session services


Get Open, High, Low, Close, Volume, and numEvents for a ticker.
        Return pandas DataFrame

        Parameters
        ----------
        ticker: string
            String corresponding to ticker
        start_datetime: string
            UTC datetime in format YYYY-mm-ddTHH:MM:SS
        end_datetime: string
            UTC datetime in format YYYY-mm-ddTHH:MM:SS
        event_type: string {TRADE, BID, ASK, BID_BEST, ASK_BEST, BEST_BID,
                           BEST_ASK}
            Requested data event type
        interval: int {1... 1440}
            Length of time bars
        elms: list of tuples
            List of tuples where each tuple corresponds to the other elements
            to be set. Refer to the IntradayBarRequest section in the
            'Services & schemas reference guide' for more info on these values


Enqueue task with specified data.


This method is a good one to extend if you want to create a queue which always applies an extra predicate.


Designed to be passed as the default kwarg in simplejson.dumps.  Serializes dates and datetimes to ISO strings.


Returns a new connection to the database.


Run a set of InsertWorkers and record their performance.


Used for development only


Returns the number of connections cached by the pool.


OperationalError's are emitted by the _mysql library for
        almost every error code emitted by MySQL.  Because of this we
        verify that the error is actually a connection error before
        terminating the connection and firing off a PoolConnectionException


Build a simple expression ready to be added onto another query.

    >>> simple_expression(joiner=' AND ', name='bob', role='admin')
    "`name`=%(_QB_name)s AND `name`=%(_QB_role)s", { '_QB_name': 'bob', '_QB_role': 'admin' }


Build a update query.

    >>> update('foo_table', a=5, b=2)
    "UPDATE `foo_table` SET `a`=%(_QB_a)s, `b`=%(_QB_b)s", { '_QB_a': 5, '_QB_b': 2 }


Connect to the database specified


Start a step.


Stop a step.


load steps -> basically load all the datetime isoformats into datetimes


Assemble one EVM instruction from its textual representation.

        :param asmcode: assembly code for one instruction
        :type asmcode: str
        :param pc: program counter of the instruction(optional)
        :type pc: int
        :param fork: fork name (optional)
        :type fork: str
        :return: An Instruction object
        :rtype: Instruction

        Example use::

            >>> print assemble_one('LT')


Assemble a sequence of textual representation of EVM instructions

        :param asmcode: assembly code for any number of instructions
        :type asmcode: str
        :param pc: program counter of the first instruction(optional)
        :type pc: int
        :param fork: fork name (optional)
        :type fork: str
        :return: An generator of Instruction objects
        :rtype: generator[Instructions]

        Example use::

            >>> assemble_one('''PUSH1 0x60\n \
                            PUSH1 0x40\n \
                            MSTORE\n \
                            PUSH1 0x2\n \
                            PUSH2 0x108\n \
                            PUSH1 0x0\n \
                            POP\n \
                            SSTORE\n \
                            PUSH1 0x40\n \
                            MLOAD\n \
                            ''')


Disassemble a single instruction from a bytecode

        :param bytecode: the bytecode stream
        :type bytecode: str | bytes | bytearray | iterator
        :param pc: program counter of the instruction(optional)
        :type pc: int
        :param fork: fork name (optional)
        :type fork: str
        :return: an Instruction object
        :rtype: Instruction

        Example use::

            >>> print disassemble_one('\x60\x10')


Disassemble all instructions in bytecode

        :param bytecode: an evm bytecode (binary)
        :type bytecode: str | bytes | bytearray | iterator
        :param pc: program counter of the first instruction(optional)
        :type pc: int
        :param fork: fork name (optional)
        :type fork: str
        :return: An generator of Instruction objects
        :rtype: list[Instruction]

        Example use::

            >>> for inst in disassemble_all(bytecode):
            ...    print(instr)

            ...
            PUSH1 0x60
            PUSH1 0x40
            MSTORE
            PUSH1 0x2
            PUSH2 0x108
            PUSH1 0x0
            POP
            SSTORE
            PUSH1 0x40
            MLOAD


Convert block number to fork name.

        :param block_number: block number
        :type block_number: int
        :return: fork name
        :rtype: str

        Example use::

            >>> block_to_fork(0)
            ...
            "frontier"
            >>> block_to_fork(4370000)
            ...
            "byzantium"
            >>> block_to_fork(4370001)
            ...
            "byzantium"


Disconnects from the websocket connection and joins the Thread.

        :return:


Issues a reconnection by setting the reconnect_required event.

        :return:


Creates a websocket connection.

        :return:


Handles and passes received data to the appropriate handlers.

        :return:


Stops ping, pong and connection timers.

        :return:


Sends a ping message to the API and starts pong timers.

        :return:


Sends the given Payload to the API via the websocket connection.

        :param kwargs: payload paarameters as key=value pairs
        :return:


Unpauses the connection.

        Send a message up to client that he should re-subscribe to all
        channels.

        :return:


Distributes system messages to the appropriate handler.

        System messages include everything that arrives as a dict,
        or a list containing a heartbeat.

        :param data:
        :param ts:
        :return:


Handle INFO messages from the API and issues relevant actions.

        :param data:
        :param ts:


Handles data messages by passing them up to the client.

        :param data:
        :param ts:
        :return:


Resubscribes to all channels found in self.channel_configs.

        :param soft: if True, unsubscribes first.
        :return: None


Handles authentication responses.

        :param dtype:
        :param data:
        :param ts:
        :return:


Handles configuration messages.

        :param dtype:
        :param data:
        :param ts:
        :return:


Reset the client.

        :return:


Return a queue containing all received candles data.

        :param pair: str, Symbol pair to request data for
        :param timeframe: str
        :return: Queue()


Send configuration to websocket server

        :param decimals_as_strings: bool, turn on/off decimals as strings
        :param ts_as_dates: bool, decide to request timestamps as dates instead
        :param sequencing: bool, turn on sequencing
	:param ts: bool, request the timestamp to be appended to every array
               sent by the server
        :param kwargs:
        :return:


Unsubscribe to the passed pair's ticker channel.

        :param pair: str, Symbol pair to request data for
        :param kwargs:
        :return:


Subscribe to the passed pair's order book channel.

        :param pair: str, Symbol pair to request data for
        :param kwargs:
        :return:


Unsubscribe to the passed pair's order book channel.

        :param pair: str, Symbol pair to request data for
        :param kwargs:
        :return:


Subscribe to the passed pair's raw order book channel.

        :param pair: str, Symbol pair to request data for
        :param prec:
        :param kwargs:
        :return:


Unsubscribe to the passed pair's raw order book channel.

        :param pair: str, Symbol pair to request data for
        :param prec:
        :param kwargs:
        :return:


Subscribe to the passed pair's trades channel.

        :param pair: str, Symbol pair to request data for
        :param kwargs:
        :return:


Unsubscribe to the passed pair's trades channel.

        :param pair: str, Symbol pair to request data for
        :param kwargs:
        :return:


Authenticate with the Bitfinex API.

        :return:


Internal callback for device command messages, parses source device from topic string and
        passes the information on to the registered device command callback


Internal callback for gateway command messages, parses source device from topic string and
        passes the information on to the registered device command callback


Internal callback for gateway notification messages, parses source device from topic string and
        passes the information on to the registered device command callback


Register one or more new device types, each request can contain a maximum of 512KB.


Publish an event to Watson IoT Platform.

        # Parameters
        event (string): Name of this event
        msgFormat (string): Format of the data for this event
        data (dict): Data for this event
        qos (int): MQTT quality of service level to use (`0`, `1`, or `2`)
        on_publish(function): A function that will be called when receipt
           of the publication is confirmed.

        # Callback and QoS
        The use of the optional #on_publish function has different implications depending
        on the level of qos used to publish the event:

        - qos 0: the client has asynchronously begun to send the event
        - qos 1 and 2: the client has confirmation of delivery from the platform


Update an existing device


Iterate through all Connectors


List all device management extension packages


Create a new device management extension package
        In case of failure it throws APIException


Update a schema. Throws APIException on failure.


Disconnect the client from IBM Watson IoT Platform


Called when the broker responds to our connection request.

        The value of rc determines success or not:
            0: Connection successful
            1: Connection refused - incorrect protocol version
            2: Connection refused - invalid client identifier
            3: Connection refused - server unavailable
            4: Connection refused - bad username or password
            5: Connection refused - not authorised
            6-255: Currently unused.


Subscribe to device event messages

        # Parameters
        typeId (string): typeId for the subscription, optional.  Defaults to all device types (MQTT `+` wildcard)
        deviceId (string): deviceId for the subscription, optional.  Defaults to all devices (MQTT `+` wildcard)
        eventId (string): eventId for the subscription, optional.  Defaults to all events (MQTT `+` wildcard)
        msgFormat (string): msgFormat for the subscription, optional.  Defaults to all formats (MQTT `+` wildcard)
        qos (int): MQTT quality of service level to use (`0`, `1`, or `2`)

        # Returns
        int: If the subscription was successful then the return Message ID (mid) for the subscribe request
            will be returned. The mid value can be used to track the subscribe request by checking against
            the mid argument if you register a subscriptionCallback method.
            If the subscription fails then the return value will be `0`


Publish a command to a device

        # Parameters
        typeId (string) : The type of the device this command is to be published to
        deviceId (string): The id of the device this command is to be published to
        command (string) : The name of the command
        msgFormat (string) : The format of the command payload
        data (dict) : The command data
        qos (int) : The equivalent MQTT semantics of quality of service using the same constants (optional, defaults to `0`)
        on_publish (function) : A function that will be called when receipt of the publication is confirmed.  This has
            different implications depending on the qos:
            - qos 0 : the client has asynchronously begun to send the event
            - qos 1 and 2 : the client has confirmation of delivery from WIoTP


Internal callback for messages that have not been handled by any of the specific internal callbacks, these
        messages are not passed on to any user provided callback


Internal callback for device event messages, parses source device from topic string and
        passes the information on to the registerd device event callback


Internal callback for device status messages, parses source device from topic string and
        passes the information on to the registerd device status callback


Internal callback for application command messages, parses source application from topic string and
        passes the information on to the registerd applicaion status callback


Retrieves the last cached message for specified event from a specific device.


Retrieves a list of the last cached message for all events from a specific device.


Initiates a device management request, such as reboot.
        In case of failure it throws APIException


Force a flush of the index to storage. Renders index
        inaccessible.


Returns the ``k``-nearest objects to the given coordinates.

        :param coordinates: sequence or array
            This may be an object that satisfies the numpy array
            protocol, providing the index's dimension * 2 coordinate
            pairs representing the `mink` and `maxk` coordinates in
            each dimension defining the bounds of the query window.

        :param num_results: integer
            The number of results to return nearest to the given coordinates.
            If two index entries are equidistant, *both* are returned.
            This property means that :attr:`num_results` may return more
            items than specified

        :param objects: True / False / 'raw'
            If True, the nearest method will return index objects that
            were pickled when they were stored with each index entry, as
            well as the id and bounds of the index entries.
            If 'raw', it will return the object as entered into the database
            without the :class:`rtree.index.Item` wrapper.

        Example of finding the three items nearest to this one::

            >>> from rtree import index
            >>> idx = index.Index()
            >>> idx.insert(4321, (34.37, 26.73, 49.37, 41.73), obj=42)
            >>> hits = idx.nearest((0, 0, 10, 10), 3, objects=True)


Deletes items from the index with the given ``'id'`` within the
        specified coordinates.

        :param id: long integer
            A long integer that is the identifier for this index entry.  IDs
            need not be unique to be inserted into the index, and it is up
            to the user to ensure they are unique if this is a requirement.

        :param coordinates: sequence or array
            Dimension * 2 coordinate pairs, representing the min
            and max coordinates in each dimension of the item to be
            deleted from the index. Their ordering will depend on the
            index's :attr:`interleaved` data member.
            These are not the coordinates of a space containing the
            item, but those of the item itself. Together with the
            id parameter, they determine which item will be deleted.
            This may be an object that satisfies the numpy array protocol.

        Example::

            >>> from rtree import index
            >>> idx = index.Index()
            >>> idx.delete(4321,
            ...            (34.3776829412, 26.7375853734, 49.3776829412,
            ...             41.7375853734))


Must be overridden. Must return a string with the loaded data.


Deletes the item from the container within the specified
        coordinates.

        :param obj: object
            Any object.

        :param coordinates: sequence or array
            Dimension * 2 coordinate pairs, representing the min
            and max coordinates in each dimension of the item to be
            deleted from the index. Their ordering will depend on the
            index's :attr:`interleaved` data member.
            These are not the coordinates of a space containing the
            item, but those of the item itself. Together with the
            id parameter, they determine which item will be deleted.
            This may be an object that satisfies the numpy array protocol.

        Example::

            >>> from rtree import index
            >>> idx = index.RtreeContainer()
            >>> idx.delete(object(),
            ...            (34.3776829412, 26.7375853734, 49.3776829412,
            ...             41.7375853734))
            Traceback (most recent call last):
             ...
            IndexError: object is not in the index


Define delay adjustment policy


Convert string into camel case.

    Args:
        string: String to convert.

    Returns:
        string: Camel case string.


Convert string into capital case.
    First letters will be uppercase.

    Args:
        string: String to convert.

    Returns:
        string: Capital case string.


Convert string into spinal case.
    Join punctuation with backslash.

    Args:
        string: String to convert.

    Returns:
        string: Spinal cased string.


Convert string into sentence case.
    First letter capped and each punctuations are joined with space.

    Args:
        string: String to convert.

    Returns:
        string: Sentence cased string.


Convert string into snake case.
    Join punctuation with underscore

    Args:
        string: String to convert.

    Returns:
        string: Snake cased string.


Attempt an import of the specified application


Initializes the Flask application with Common.


Return a PIL Image instance cropped from `image`.

        Image has an aspect ratio provided by dividing `width` / `height`),
        sized down to `width`x`height`. Any 'excess pixels' are trimmed away
        in respect to the pixel of `image` that corresponds to `ppoi` (Primary
        Point of Interest).

        `image`: A PIL Image instance
        `width`: Integer, width of the image to return (in pixels)
        `height`: Integer, height of the image to return (in pixels)
        `ppoi`: A 2-tuple of floats with values greater than 0 and less than 1
                These values are converted into a cartesian coordinate that
                signifies the 'center pixel' which the crop will center on
                (to trim the excess from the 'long side').

        Determines whether to trim away pixels from either the left/right or
        top/bottom sides by comparing the aspect ratio of `image` vs the
        aspect ratio of `width`x`height`.

        Will trim from the left/right sides if the aspect ratio of `image`
        is greater-than-or-equal-to the aspect ratio of `width`x`height`.

        Will trim from the top/bottom sides if the aspect ration of `image`
        is less-than the aspect ratio or `width`x`height`.

        Similar to Kevin Cazabon's ImageOps.fit method but uses the
        ppoi value as an absolute centerpoint (as opposed as a
        percentage to trim off the 'long sides').


Return a BytesIO instance of `image` that fits in a bounding box.

        Bounding box dimensions are `width`x`height`.


Return a BytesIO instance of `image` with inverted colors.


Ensure data is prepped properly before handing off to ImageField.


Process the field's placeholder image.

        Ensures the placeholder image has been saved to the same storage class
        as the field in a top level folder with a name specified by
        settings.VERSATILEIMAGEFIELD_SETTINGS['placeholder_directory_name']

        This should be called by the VersatileImageFileDescriptor __get__.
        If self.placeholder_image_name is already set it just returns right away.


Return field's value just before saving.


Update field's ppoi field, if defined.

        This method is hooked up this field's pre_save method to update
        the ppoi immediately before the model instance (`instance`)
        it is associated with is saved.

        This field's ppoi can be forced to update with force=True,
        which is how VersatileImageField.pre_save calls this method.


Handle data sent from MultiValueField forms that set ppoi values.

        `instance`: The model instance that is being altered via a form
        `data`: The data sent from the form to this field which can be either:
        * `None`: This is unset data from an optional field
        * A two-position tuple: (image_form_data, ppoi_data)
            * `image_form-data` options:
                * `None` the file for this field is unchanged
                * `False` unassign the file form the field
            * `ppoi_data` data structure:
                * `%(x_coordinate)sx%(y_coordinate)s': The ppoi data to
                  assign to the unchanged file


Unregister the FilteredImage subclass currently assigned to attr_name.

        If a FilteredImage subclass isn't already registered to filters.
        `attr_name` NotRegistered will raise.


Return the appropriate URL.

        URL is constructed based on these field conditions:
            * If empty (not `self.name`) and a placeholder is defined, the
              URL to the placeholder is returned.
            * Otherwise, defaults to vanilla ImageFieldFile behavior.


Return the location where filtered images are stored.


Return the location where sized images are stored.


Return the location where filtered + sized images are stored.


Preprocess an image.

        An API hook for image pre-processing. Calls any image format specific
        pre-processors (if defined). I.E. If `image_format` is 'JPEG', this
        method will look for a method named `preprocess_JPEG`, if found
        `image` will be passed to it.

        Arguments:
            * `image`: a PIL Image instance
            * `image_format`: str, a valid PIL format (i.e. 'JPEG' or 'GIF')

        Subclasses should return a 2-tuple:
            * [0]: A PIL Image instance.
            * [1]: A dictionary of additional keyword arguments to be used
                   when the instance is saved. If no additional keyword
                   arguments, return an empty dict ({}).


Receive a PIL Image instance of a JPEG and returns 2-tuple.

        Args:
            * [0]: Image instance, converted to RGB
            * [1]: Dict with a quality key (mapped to the value of `QUAL` as
                   defined by the `VERSATILEIMAGEFIELD_JPEG_RESIZE_QUALITY`
                   setting)


Return a PIL Image instance stored at `path_to_image`.


Return PPOI value as a string.


Create a resized image.

        `path_to_image`: The path to the image with the media directory to
                         resize. If `None`, the
                         VERSATILEIMAGEFIELD_PLACEHOLDER_IMAGE will be used.
        `save_path_on_storage`: Where on self.storage to save the resized image
        `width`: Width of resized image (int)
        `height`: Desired height of resized image (int)
        `filename_key`: A string that will be used in the sized image filename
                        to signify what operation was done to it.
                        Examples: 'crop' or 'scale'


Return a `path_to_image` location on `storage` as dictated by `width`, `height`
    and `filename_key`


Return the 'filtered path'


Validate a list of size keys.

    `sizes`: An iterable of 2-tuples, both strings. Example:
    [
        ('large', 'url'),
        ('medium', 'crop__400x400'),
        ('small', 'thumbnail__100x100')
    ]


Build a URL from `image_key`.


Takes a raw `Instruction` and translates it into a human readable text
    representation. As of writing, the text representation for WASM is not yet
    standardized, so we just emit some generic format.


Takes a `FunctionBody` and optionally a `FunctionType`, yielding the string
    representation of the function line by line. The function type is required
    for formatting function parameter and return value information.


Decodes raw bytecode, yielding `Instruction`s.


Deprecates a function, printing a warning on the first usage.


Checks the validity of the input.

        In case of an invalid input throws ValueError.


Helper method that returns the index of the string based on node's
        starting index


Returns the Largest Common Substring of Strings provided in stringIdxs.
        If stringIdxs is not provided, the LCS of all strings is returned.

        ::param stringIdxs: Optional: List of indexes of strings.


Helper method returns the starting indexes of strings in GST


Helper method, returns the edge label between a node and it's parent


Generator of unique terminal symbols used for building the Generalized Suffix Tree.
        Unicode Private Use Area U+E000..U+F8FF is used to ensure that terminal symbols
        are not part of the input string.


connect to the server


Parse read a response from the AGI and parse it.

        :return dict: The AGI response parsed into a dict.


Parse AGI results using Regular expression.

    AGI Result examples::

        100 result=0 Trying...

        200 result=0

        200 result=-1

        200 result=132456

        200 result= (timeout)

        510 Invalid or unknown command

        520-Invalid command syntax. Proper usage follows:
        int() argument must be a string, a bytes-like object or a number, not
        'NoneType'

        HANGUP


Mostly used for unit testing. Allow to use a static uuid and reset
        all counter


Mostly used for debugging


Returns data from a package directory.
    'path' should be an absolute path.


Create a graph of constraints for both must- and cannot-links


Translates a regular Scikit-Learn estimator or pipeline to a PMML pipeline.

	Parameters:
	----------
	obj: BaseEstimator
		The object.

	active_fields: list of strings, optional
		Feature names. If missing, "x1", "x2", .., "xn" are assumed.

	target_fields: list of strings, optional
		Label name(s). If missing, "y" is assumed.


Converts a fitted Scikit-Learn pipeline to PMML.

	Parameters:
	----------
	pipeline: PMMLPipeline
		The pipeline.

	pmml: string
		The path to where the PMML document should be stored.

	user_classpath: list of strings, optional
		The paths to JAR files that provide custom Transformer, Selector and/or Estimator converter classes.
		The JPMML-SkLearn classpath is constructed by appending user JAR files to package JAR files.

	with_repr: boolean, optional
		If true, insert the string representation of pipeline into the PMML document.

	debug: boolean, optional
		If true, print information about the conversion process.

	java_encoding: string, optional
		The character encoding to use for decoding Java output and error byte streams.


Returns an instance of the formset


If the formset is valid, save the associated models.


Handles POST requests, instantiating a formset instance with the passed
        POST variables and then checked for validity.


Overrides construct_formset to attach the model class as
        an attribute of the returned formset instance.


Returns the inline formset instances


Handles GET requests and instantiates a blank version of the form and formsets.


Handles POST requests, instantiating a form and formset instances with the passed
        POST variables and then checked for validity.


If `inlines_names` has been defined, add each formset to the context under
        its corresponding entry in `inlines_names`


Returns the start date for a model instance


Returns an integer representing the first day of the week.

        0 represents Monday, 6 represents Sunday.


Returns a queryset of models for the month requested


Injects variables necessary for rendering the calendar into the context.

        Variables added are: `calendar`, `weekdays`, `month`, `next_month` and `previous_month`.


Get primary key properties for a SQLAlchemy model.

    :param model: SQLAlchemy model class


Deserialize a serialized value to a model instance.

        If the parent schema is transient, create a new (transient) instance.
        Otherwise, attempt to find an existing instance in the database.
        :param value: The value to deserialize.


Deserialize data to internal representation.

        :param session: Optional SQLAlchemy session.
        :param instance: Optional existing instance to modify.
        :param transient: Optional switch to allow transient instantiation.


Deletes old stellar tables that are not used anymore


Takes a snapshot of the database


Returns a list of snapshots


Removes a snapshot


Renames a snapshot


Replaces a snapshot


Updates indexes after each epoch for shuffling


Defines the default function for cleaning text.

    This function operates over a list.


Apply function to list of elements.

    Automatically determines the chunk size.


Analyze document length statistics for padding strategy


Return a new Colorful object with the given color config.


Parse the given rgb.txt file into a Python dict.

    See https://en.wikipedia.org/wiki/X11_color_names for more information

    :param str path: the path to the X11 rgb.txt file


Sanitze the given color palette so it can
    be safely used by Colorful.

    It will convert colors specified in hex RGB to
    a RGB channel triplet.


Detect what color palettes are supported.
    It'll return a valid color mode to use
    with colorful.

    :param dict env: the environment dict like returned by ``os.envion``


Convert the given hex string to a
    valid RGB channel triplet.


Check if the given hex value is a valid RGB color

    It should match the format: [0-9a-fA-F]
    and be of length 3 or 6.


Translate the given color name to a valid
    ANSI escape code.

    :parma str colorname: the name of the color to resolve
    :parma str offset: the offset for the color code
    :param int colormode: the color mode to use. See ``translate_rgb_to_ansi_code``
    :parma dict colorpalette: the color palette to use for the color name mapping

    :returns str: the color as ANSI escape code

    :raises ColorfulError: if the given color name is invalid


Resolve the given modifier name to a valid
    ANSI escape code.

    :param str modifiername: the name of the modifier to resolve
    :param int colormode: the color mode to use. See ``translate_rgb_to_ansi_code``

    :returns str: the ANSI escape code for the modifier

    :raises ColorfulError: if the given modifier name is invalid


Translate the given style to an ANSI escape code
    sequence.

    ``style`` examples are:

    * green
    * bold
    * red_on_black
    * bold_green
    * italic_yellow_on_cyan

    :param str style: the style to translate
    :param int colormode: the color mode to use. See ``translate_rgb_to_ansi_code``
    :parma dict colorpalette: the color palette to use for the color name mapping


Style the given string according to the given
    ANSI style string.

    :param str string: the string to style
    :param tuple ansi_style: the styling string returned by ``translate_style``
    :param int colormode: the color mode to use. See ``translate_rgb_to_ansi_code``

    :returns: a string containing proper ANSI sequence


Use a predefined style as color palette

        :param str style_name: the name of the style


Format the given string with the given ``args`` and ``kwargs``.
        The string can contain references to ``c`` which is provided by
        this colorful object.

        :param str string: the string to format


Get data from the USB device.


Get device humidity reading.

        Params:
        - sensors: optional list of sensors to get a reading for, examples:
          [0,] - get reading for sensor 0
          [0, 1,] - get reading for sensors 0 and 1
          None - get readings for all sensors


Read data from device.


Update, rolling back on failure.


Create a new temporary file and write some initial text to it.

    :param text: the text to write to the temp file
    :type text: str
    :returns: the file name of the newly created temp file
    :rtype: str


Get a list of contacts from one or more address books.

    :param address_books: the address books to search
    :type address_books: list(address_book.AddressBook)
    :param query: a search query to select contacts
    :type quer: str
    :param method: the search method, one of "all", "name" or "uid"
    :type method: str
    :param reverse: reverse the order of the returned contacts
    :type reverse: bool
    :param group: group results by address book
    :type group: bool
    :param sort: the field to use for sorting, one of "first_name", "last_name"
    :type sort: str
    :returns: contacts from the address_books that match the query
    :rtype: list(CarddavObject)


Merge the parsed arguments from argparse into the config object.

    :param args: the parsed command line arguments
    :type args: argparse.Namespace
    :param config: the parsed config file
    :type config: config.Config
    :returns: the merged config object
    :rtype: config.Config


Load all address books with the given names from the config.

    :param names: the address books to load
    :type names: list(str)
    :param config: the config instance to use when looking up address books
    :type config: config.Config
    :param search_queries: a mapping of address book names to search queries
    :type search_queries: dict
    :yields: the loaded address books
    :ytype: addressbook.AddressBook


Prepare the search query string from the given command line args.

    Each address book can get a search query string to filter vcards befor
    loading them.  Depending on the question if the address book is used for
    source or target searches different regexes have to be combined into one
    search string.

    :param args: the parsed command line
    :type args: argparse.Namespace
    :returns: a dict mapping abook names to their loading queries, if the query
        is None it means that all cards should be loaded
    :rtype: dict(str:str or None)


Print a phone application friendly contact table.

    :param search_terms: used as search term to filter the contacts before
        printing
    :type search_terms: str
    :param vcard_list: the vcards to search for matching entries which should
        be printed
    :type vcard_list: list of carddav_object.CarddavObject
    :param parsable: machine readable output: columns devided by tabulator (\t)
    :type parsable: bool
    :returns: None
    :rtype: None


Print a user friendly contacts table.

    :param vcard_list: the vcards to print
    :type vcard_list: list of carddav_object.CarddavObject
    :param parsable: machine readable output: columns devided by tabulator (\t)
    :type parsable: bool
    :returns: None
    :rtype: None


Modify a contact in an external editor.

    :param selected_vcard: the contact to modify
    :type selected_vcard: carddav_object.CarddavObject
    :param input_from_stdin_or_file: new data from stdin (or a file) that
        should be incorperated into the contact, this should be a yaml
        formatted string
    :type input_from_stdin_or_file: str
    :param open_editor: whether to open the new contact in the edior after
        creation
    :type open_editor: bool
    :returns: None
    :rtype: None


Remove a contact from the addressbook.

    :param selected_vcard: the contact to delete
    :type selected_vcard: carddav_object.CarddavObject
    :param force: delete without confirmation
    :type force: bool
    :returns: None
    :rtype: None


Open the vcard file for a contact in an external editor.

    :param selected_vcard: the contact to edit
    :type selected_vcard: carddav_object.CarddavObject
    :param editor: the eitor command to use
    :type editor: str
    :returns: None
    :rtype: None


Merge two contacts into one.

    :param vcard_list: the vcards from which to choose contacts for mergeing
    :type vcard_list: list of carddav_object.CarddavObject
    :param selected_address_books: the addressbooks to use to find the target
        contact
    :type selected_address_books: list(addressbook.AddressBook)
    :param search_terms: the search terms to find the target contact
    :type search_terms: str
    :param target_uid: the uid of the target contact or empty
    :type target_uid: str
    :returns: None
    :rtype: None


Find the name of the action for the supplied alias.  If no action is
        asociated with the given alias, None is returned.

        :param alias: the alias to look up
        :type alias: str
        :rturns: the name of the corresponding action or None
        :rtype: str or NoneType


Convert the named field to bool.

        The current value should be one of the strings "yes" or "no".  It will
        be replaced with its boolean counterpart.  If the field is not present
        in the config object, the default value is used.

        :param config: the config section where to set the option
        :type config: configobj.ConfigObj
        :param name: the name of the option to convert
        :type name: str
        :param default: the default value to use if the option was not
            previously set
        :type default: bool
        :returns: None


Use this if you want to create a new contact from user input.


Get some part of the "N" entry in the vCard as a list

        :param part: the name to get e.g. "prefix" or "given"
        :type part: str
        :returns: a list of entries for this name part
        :rtype: list(str)


categories variable must be a list


Parse type value of phone numbers, email and post addresses.

        :param types: list of type values
        :type types: list(str)
        :param value: the corresponding label, required for more verbose
            exceptions
        :type value: str
        :param supported_types: all allowed standard types
        :type supported_types: list(str)
        :returns: tuple of standard and custom types and pref integer
        :rtype: tuple(list(str), list(str), int)


converts list to string recursively so that nested lists are supported

    :param input: a list of strings and lists of strings (and so on recursive)
    :type input: list
    :param delimiter: the deimiter to use when joining the items
    :type delimiter: str
    :returns: the recursively joined list
    :rtype: str


Convert string to date object.

    :param input: the date string to parse
    :type input: str
    :returns: the parsed datetime object
    :rtype: datetime.datetime


Calculate the minimum length of initial substrings of uid1 and uid2
        for them to be different.

        :param uid1: first uid to compare
        :type uid1: str
        :param uid2: second uid to compare
        :type uid2: str
        :returns: the length of the shortes unequal initial substrings
        :rtype: int


Search in all fields for contacts matching query.

        :param query: the query to search for
        :type query: str
        :yields: all found contacts
        :rtype: generator(carddav_object.CarddavObject)


Search in the name filed for contacts matching query.

        :param query: the query to search for
        :type query: str
        :yields: all found contacts
        :rtype: generator(carddav_object.CarddavObject)


Search for contacts with a matching uid.

        :param query: the query to search for
        :type query: str
        :yields: all found contacts
        :rtype: generator(carddav_object.CarddavObject)


Search this address book for contacts matching the query.

        The method can be one of "all", "name" and "uid".  The backend for this
        address book migth be load()ed if needed.

        :param query: the query to search for
        :type query: str
        :param method: the type of fileds to use when seaching
        :type method: str
        :returns: all found contacts
        :rtype: list(carddav_object.CarddavObject)


Create a dictionary of shortend UIDs for all contacts.

        All arguments are only used if the address book is not yet initialized
        and will just be handed to self.load().

        :param query: see self.load()
        :type query: str
        :returns: the contacts mapped by the shortes unique prefix of their UID
        :rtype: dict(str: CarddavObject)


Get the shortend UID for the given UID.

        :param uid: the full UID to shorten
        :type uid: str
        :returns: the shortend uid or the empty string
        :rtype: str


Load all vcard files in this address book from disk.

        If a search string is given only files which contents match that will
        be loaded.

        :param query: a regular expression to limit the results
        :type query: str
        :param search_in_source_files: apply search regexp directly on the .vcf files to speed up parsing (less accurate)
        :type search_in_source_files: bool
        :returns: the number of successfully loaded cards and the number of
            errors
        :rtype: int, int
        :throws: AddressBookParseError


Create the JSON for configuring arthur to collect data

        https://github.com/grimoirelab/arthur#adding-tasks
        Sample for git:

        {
        "tasks": [
            {
                "task_id": "arthur.git",
                "backend": "git",
                "backend_args": {
                    "gitpath": "/tmp/arthur_git/",
                    "uri": "https://github.com/grimoirelab/arthur.git"
                },
                "category": "commit",
                "archive_args": {
                    "archive_path": '/tmp/test_archives',
                    "fetch_from_archive": false,
                    "archive_after": None
                },
                "scheduler_args": {
                    "delay": 10
                }
            }
        ]
        }


Return the GitHub SHA for a file in the repository


Execute the merge identities phase

    :param config: a Mordred config object


Execute the panels phase

    :param config: a Mordred config object


Config logging level output output


Get params to execute the micro-mordred


Upload a panel to Elasticsearch if it does not exist yet.

        If a list of data sources is specified, upload only those
        elements (visualizations, searches) that match that data source.

        :param panel_file: file name of panel (dashobard) to upload
        :param data_sources: list of data sources
        :param strict: only upload a dashboard if it is newer than the one already existing


Upload to Kibiter the title for the dashboard.

        The title is shown on top of the dashboard menu, and is Usually
        the name of the project being dashboarded.
        This is done only for Kibiter 6.x.

        :param kibiter_major: major version of kibiter


Create the menu definition to access the panels in a dashboard.

        :param          menu: dashboard menu to upload
        :param kibiter_major: major version of kibiter


Remove existing menu for dashboard, if any.

        Usually, we remove the menu before creating a new one.

        :param kibiter_major: major version of kibiter


Get the menu entries from the panel definition


Order the dashboard menu


Compose projects.json only for mbox, but using the mailing_lists lists

    change: 'https://dev.eclipse.org/mailman/listinfo/emft-dev'
    to: 'emfg-dev /home/bitergia/mboxes/emft-dev.mbox/emft-dev.mbox

    :param projects: projects.json
    :return: projects.json with mbox


Compose projects.json for git

    We need to replace '/c/' by '/gitroot/' for instance

    change: 'http://git.eclipse.org/c/xwt/org.eclipse.xwt.git'
    to: 'http://git.eclipse.org/gitroot/xwt/org.eclipse.xwt.git'

    :param projects: projects.json
    :param data: eclipse JSON
    :return: projects.json with git


Compose projects.json for mailing lists

    At upstream has two different key for mailing list: 'mailings_lists' and 'dev_list'
    The key 'mailing_lists' is an array with mailing lists
    The key 'dev_list' is a dict with only one mailing list

    :param projects: projects.json
    :param data: eclipse JSON
    :return: projects.json with mailing_lists


Compose projects.json for github

    :param projects: projects.json
    :param data: eclipse JSON
    :return: projects.json with github


Compose the projects JSON file only with the projects name

    :param projects: projects.json
    :param data: eclipse JSON with the origin format
    :return: projects.json with titles


Compose projects.json with all data sources

    :param projects: projects.json
    :param data: eclipse JSON
    :return: projects.json with all data sources


Execute autorefresh for areas of code study if configured


Execute the studies configured for the current backend


Retain the identities in SortingHat based on the `retention_time`
        value declared in the setup.cfg.

        :param retention_time: maximum number of minutes wrt the current date to retain the SortingHat data


return list with the repositories for a backend_section


Convert from eclipse projects format to grimoire projects json format


Change a param in the config


Get Elasticsearch version.

        Get the version of Elasticsearch. This is useful because
        Elasticsearch and Kibiter are paired (same major version for 5, 6).

        :param url: Elasticseearch url hosting Kibiter indices
        :returns:   major version, as string


Start a task manager per backend to complete the tasks.

        :param task_cls: list of tasks classes to be executed
        :param big_delay: seconds before global tasks are executed, should be days usually
        :param small_delay: seconds before backend tasks are executed, should be minutes
        :param wait_for_threads: boolean to set when threads are infinite or
                                should be synchronized in a meeting point


Tasks that should be done just one time


Validates the provided config to make sure all the required fields are
        there.


Customize the message format based on the log level.


Initialize the dictionary of architectures for assembling via keystone


Sys.out replacer, by default with stderr.

  Use it like this:
  with replace_print_with(fileobj):
    print "hello"  # writes to the file
  print "done"  # prints to stdout

  Args:
    fileobj: a file object to replace stdout.

  Yields:
    The printer.


Compact a list of integers into a comma-separated string of intervals.

  Args:
    value_list: A list of sortable integers such as a list of numbers

  Returns:
    A compact string representation, such as "1-5,8,12-15"


Get a storage client using the provided credentials or defaults.


Load context from a text file in gcs.

  Args:
    gcs_file_path: The target file path; should have the 'gs://' prefix.
    credentials: Optional credential to be used to load the file from gcs.

  Returns:
    The content of the text file as a string.


Check whether the file exists, in GCS.

  Args:
    gcs_file_path: The target file path; should have the 'gs://' prefix.
    credentials: Optional credential to be used to load the file from gcs.

  Returns:
    True if the file's there.


True iff an object exists matching the input GCS pattern.

  The GCS pattern must be a full object reference or a "simple pattern" that
  conforms to the dsub input and output parameter restrictions:

    * No support for **, ? wildcards or [] character ranges
    * Wildcards may only appear in the file name

  Args:
    file_pattern: eg. 'gs://foo/ba*'
    credentials: Optional credential to be used to load the file from gcs.

  Raises:
    ValueError: if file_pattern breaks the rules.

  Returns:
    True iff a file exists that matches that pattern.


True if each output contains at least one file or no output specified.


Return a dict object representing a pipeline input argument.


Return a multi-line string of the full pipeline docker command.


Builds pipeline args for execution.

    Args:
      project: string name of project.
      script: Body of the script to execute.
      job_params: dictionary of values for labels, envs, inputs, and outputs
          for this job.
      task_params: dictionary of values for labels, envs, inputs, and outputs
          for this task.
      reserved_labels: dictionary of reserved labels (e.g. task-id,
          task-attempt)
      preemptible: use a preemptible VM for the job
      logging_uri: path for job logging output.
      scopes: list of scope.
      keep_alive: Seconds to keep VM alive on failure

    Returns:
      A nested dictionary with one entry under the key pipelineArgs containing
      the pipeline arguments.


Convert the integer UTC time value into a local datetime.


Returns a Pipeline objects for the job.


Kills the operations associated with the specified job or job.task.

    Args:
      user_ids: List of user ids who "own" the job(s) to cancel.
      job_ids: List of job_ids to cancel.
      task_ids: List of task-ids to cancel.
      labels: List of LabelParam, each must match the job(s) to be canceled.
      create_time_min: a timezone-aware datetime value for the earliest create
                       time of a task, inclusive.
      create_time_max: a timezone-aware datetime value for the most recent
                       create time of a task, inclusive.

    Returns:
      A list of tasks canceled and a list of error messages.


Returns the most relevant status string and last updated date string.

    This string is meant for display only.

    Returns:
      A printable status string and date string.


Create a task name from a job-id, task-id, and task-attempt.

  Task names are used internally by dsub as well as by the docker task runner.
  The name is formatted as "<job-id>.<task-id>[.task-attempt]". Task names
  follow formatting conventions allowing them to be safely used as a docker
  name.

  Args:
    job_id: (str) the job ID.
    task_id: (str) the task ID.
    task_attempt: (int) the task attempt.

  Returns:
    a task name string.


Rewrite string so that all characters are valid in a docker name suffix.


Return a tuple for sorting 'most recent first'.


Determine if the provided time is within the range, inclusive.


Return a Task object with this task's info.


Returns a command to delocalize logs.

    Args:
      logging_path: location of log files.
      user_project: name of the project to be billed for the request.

    Returns:
      eg. 'gs://bucket/path/myfile' or 'gs://bucket/script-foobar-12'


The local dir for staging files for that particular task.


Returns a command that will stage recursive inputs.


Returns a directory or file path to be the target for "gsutil cp".

    If the filename contains a wildcard, then the target path must
    be a directory in order to ensure consistency whether the source pattern
    contains one or multiple files.

    Args:
      local_file_path: A full path terminating in a file or a file wildcard.

    Returns:
      The path to use as the "gsutil cp" target.


Returns a command that will stage inputs.


Get the dsub version out of the _dsub_version.py source file.

  Setup.py should not import dsub version from dsub directly since ambiguity in
  import order could lead to an old version of dsub setting the version number.
  Parsing the file directly is simpler than using import tools (whose interface
  varies between python 2.7, 3.4, and 3.5).

  Returns:
    string of dsub version.

  Raises:
    ValueError: if the version is not found.


Return a dict with variables for the 'prepare' action.


Return a dict with variables for the 'localization' action.


Return a dict with variables for the 'delocalization' action.


Returns a dictionary of for the user container environment.


Returns the status of this operation.

    Raises:
      ValueError: if the operation status cannot be determined.

    Returns:
      A printable status string (RUNNING, SUCCESS, CANCELED or FAILURE).


Returns the most relevant status string and failed action.

    This string is meant for display only.

    Returns:
      A printable status string and name of failed action (if any).


Rounds ram up to the nearest multiple of _MEMORY_MULTIPLE.


Returns a custom machine type string.


Build a VirtualMachine object for a Pipeline request.

  Args:
    network (dict): Network details for the pipeline to run in.
    machine_type (str): GCE Machine Type string for the pipeline.
    preemptible (bool): Use a preemptible VM for the job.
    service_account (dict): Service account configuration for the VM.
    boot_disk_size_gb (int): Boot disk size in GB.
    disks (list[dict]): List of disks to mount.
    accelerators (list[dict]): List of accelerators to attach to the VM.
    labels (dict[string, string]): Labels for the VM.
    cpu_platform (str): The CPU platform to request.
    nvidia_driver_version (str): The NVIDIA driver version to use when attaching
      an NVIDIA GPU accelerator.

  Returns:
    An object representing a VirtualMachine.


Build an Action object for a Pipeline request.

  Args:
    name (str): An optional name for the container.
    image_uri (str): The URI to pull the container image from.
    commands (List[str]): commands and arguments to run inside the container.
    entrypoint (str): overrides the ENTRYPOINT specified in the container.
    environment (dict[str,str]): The environment to pass into the container.
    pid_namespace (str): The PID namespace to run the action inside.
    flags (str): Flags that control the execution of this action.
    port_mappings (dict[int, int]): A map of container to host port mappings for
      this container.
    mounts (List): A list of mounts to make available to the action.
    labels (dict[str]): Labels to associate with the action.

  Returns:
    An object representing an Action resource.


Returns a provider for job submission requests.


Add provider required arguments epilog message, parse, and validate.


A string with the arguments to point dstat to the same provider+project.


Returns a URI with placeholders replaced by metadata values.


Inserts task metadata into the logging URI.

  The core behavior is inspired by the Google Pipelines API:
    (1) If a the uri ends in ".log", then that is the logging path.
    (2) Otherwise, the uri is treated as "directory" for logs and a filename
        needs to be automatically generated.

  For (1), if the job is a --tasks job, then the {task-id} is inserted
  before ".log".

  For (2), the file name generated is {job-id}, or for --tasks jobs, it is
  {job-id}.{task-id}.

  In both cases .{task-attempt} is inserted before .log for --retries jobs.

  In addition, full task metadata substitution is supported. The URI
  may include substitution strings such as
  "{job-id}", "{task-id}", "{job-name}", "{user-id}", and "{task-attempt}".

  Args:
    uri: User-specified logging URI which may contain substitution fields.
    job_metadata: job-global metadata.
    task_metadata: tasks-specific metadata.

  Returns:
    The logging_uri formatted as described above.


Validated google-v2 arguments.