U MZf0 @stdZddlmZddlZddlmZmZddlZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZddlZddlZddlmZddlmZmZdd l m!Z!ddl"m#m$Z%dd l&m'Z'dd l(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5dd l6m7Z8dd l9m:Z:m;Z;ddlZ>m?Z?m@Z@ddlAmBZBddlCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLddlMmNZNmOZOddlPmQZQmRZRddlSmTZTddlUmVZVmWZWmXZXmYZYmZZZm[Z[m\Z\ddl]m^Z^m_Z_ddl`mambZcddldmeZeddlfmgZgddlhmiZimjZjmkZkddllmmZmddlnmoZompZpddlqmrZrmsZsmtZtmuZumvZvddlwmxZxddlymzZzddl{m|Z|ddl}m~Z~mZe rdd lmZmZmZd!Zd"d#d$d%Zd&Zd'Zd(Zd)ZeGd*d+d+e^Zee ee e e ge fee e ge fee e ffZGd,d-d-e^e_e0eoZed.egd/ZGd0d1d1ee0Ze@edAd3d4d5d6d7d1d8d9d:Zd;dd?d@ZdS)Ba Provide the groupby split-apply-combine paradigm. Define the GroupBy class providing the base-class of operations. The SeriesGroupBy and DataFrameGroupBy sub-class (defined in pandas.core.groupby.generic) expose these user-facing objects to provide specific functionality. ) annotationsN)partialwraps)dedent) TYPE_CHECKINGCallableHashableIterableIteratorListLiteralMappingSequenceTypeVarUnioncastfinal)option_context) Timestamplib)rank_1d)NA) AnyArrayLike ArrayLikeAxisAxisIntDtypeObj FillnaOptions IndexLabelNDFrameTPositionalIndexer RandomStateScalarTnpt)function)AbstractMethodError DataError)Appender Substitutioncache_readonlydoc)ensure_dtype_can_hold_na) is_bool_dtypeis_float_dtype is_hashable is_integeris_integer_dtypeis_numeric_dtypeis_object_dtype is_scalarneeds_i8_conversion)isnanotna) algorithmssample)executor)BaseMaskedArray BooleanArray Categorical DatetimeArrayExtensionArray FloatingArrayTimedeltaArray) PandasObjectSelectionMixin) DataFrame)NDFrame)basenumba_ops) get_grouper)GroupByIndexingMixinGroupByNthSelector)CategoricalIndexIndex MultiIndex RangeIndex default_index)ensure_block_shape)Series)get_group_index_sorter)get_jit_argumentsmaybe_use_numba)ExpandingGroupbyExponentialMovingWindowGroupbyRollingGroupbyz See Also -------- Series.%(name)s : Apply a function %(name)s to a Series. DataFrame.%(name)s : Apply a function %(name)s to each row or column of a DataFrame. aS Apply function ``func`` group-wise and combine the results together. The function passed to ``apply`` must take a {input} as its first argument and return a DataFrame, Series or scalar. ``apply`` will then take care of combining the results back together into a single dataframe or series. ``apply`` is therefore a highly flexible grouping method. While ``apply`` is a very flexible method, its downside is that using it can be quite a bit slower than using more specific methods like ``agg`` or ``transform``. Pandas offers a wide range of method that will be much faster than using ``apply`` for their specific purposes, so try to use them before reaching for ``apply``. Parameters ---------- func : callable A callable that takes a {input} as its first argument, and returns a dataframe, a series or a scalar. In addition the callable may take positional and keyword arguments. args, kwargs : tuple and dict Optional positional and keyword arguments to pass to ``func``. Returns ------- Series or DataFrame See Also -------- pipe : Apply function to the full GroupBy object instead of to each group. aggregate : Apply aggregate function to the GroupBy object. transform : Apply function column-by-column to the GroupBy object. Series.apply : Apply a function to a Series. DataFrame.apply : Apply a function to each row or column of a DataFrame. Notes ----- .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See :ref:`gotchas.udf-mutation` for more details. Examples -------- {examples} a% >>> df = pd.DataFrame({'A': 'a a b'.split(), ... 'B': [1,2,3], ... 'C': [4,6,5]}) >>> g1 = df.groupby('A', group_keys=False) >>> g2 = df.groupby('A', group_keys=True) Notice that ``g1`` and ``g2`` have two groups, ``a`` and ``b``, and only differ in their ``group_keys`` argument. Calling `apply` in various ways, we can get different grouping results: Example 1: below the function passed to `apply` takes a DataFrame as its argument and returns a DataFrame. `apply` combines the result for each group together into a new DataFrame: >>> g1[['B', 'C']].apply(lambda x: x / x.sum()) B C 0 0.333333 0.4 1 0.666667 0.6 2 1.000000 1.0 In the above, the groups are not part of the index. We can have them included by using ``g2`` where ``group_keys=True``: >>> g2[['B', 'C']].apply(lambda x: x / x.sum()) B C A a 0 0.333333 0.4 1 0.666667 0.6 b 2 1.000000 1.0 Example 2: The function passed to `apply` takes a DataFrame as its argument and returns a Series. `apply` combines the result for each group together into a new DataFrame. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g1[['B', 'C']].apply(lambda x: x.astype(float).max() - x.min()) B C A a 1.0 2.0 b 0.0 0.0 >>> g2[['B', 'C']].apply(lambda x: x.astype(float).max() - x.min()) B C A a 1.0 2.0 b 0.0 0.0 The ``group_keys`` argument has no effect here because the result is not like-indexed (i.e. :ref:`a transform `) when compared to the input. Example 3: The function passed to `apply` takes a DataFrame as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g1.apply(lambda x: x.C.max() - x.B.min()) A a 5 b 2 dtype: int64a >>> s = pd.Series([0, 1, 2], index='a a b'.split()) >>> g1 = s.groupby(s.index, group_keys=False) >>> g2 = s.groupby(s.index, group_keys=True) From ``s`` above we can see that ``g`` has two groups, ``a`` and ``b``. Notice that ``g1`` have ``g2`` have two groups, ``a`` and ``b``, and only differ in their ``group_keys`` argument. Calling `apply` in various ways, we can get different grouping results: Example 1: The function passed to `apply` takes a Series as its argument and returns a Series. `apply` combines the result for each group together into a new Series. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g1.apply(lambda x: x*2 if x.name == 'a' else x/2) a 0.0 a 2.0 b 1.0 dtype: float64 In the above, the groups are not part of the index. We can have them included by using ``g2`` where ``group_keys=True``: >>> g2.apply(lambda x: x*2 if x.name == 'a' else x/2) a a 0.0 a 2.0 b b 1.0 dtype: float64 Example 2: The function passed to `apply` takes a Series as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g1.apply(lambda x: x.max() - x.min()) a 1 b 0 dtype: int64 The ``group_keys`` argument has no effect here because the result is not like-indexed (i.e. :ref:`a transform `) when compared to the input. >>> g2.apply(lambda x: x.max() - x.min()) a 1 b 0 dtype: int64)templatedataframe_examplesZseries_examplesa Compute {fname} of group values. Parameters ---------- numeric_only : bool, default {no} Include only float, int, boolean columns. .. versionchanged:: 2.0.0 numeric_only no longer accepts ``None``. min_count : int, default {mc} The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame Computed {fname} of values within each group. aF Apply a ``func`` with arguments to this %(klass)s object and return its result. Use `.pipe` when you want to improve readability by chaining together functions that expect Series, DataFrames, GroupBy or Resampler objects. Instead of writing >>> h(g(f(df.groupby('group')), arg1=a), arg2=b, arg3=c) # doctest: +SKIP You can write >>> (df.groupby('group') ... .pipe(f) ... .pipe(g, arg1=a) ... .pipe(h, arg2=b, arg3=c)) # doctest: +SKIP which is much more readable. Parameters ---------- func : callable or tuple of (callable, str) Function to apply to this %(klass)s object or, alternatively, a `(callable, data_keyword)` tuple where `data_keyword` is a string indicating the keyword of `callable` that expects the %(klass)s object. args : iterable, optional Positional arguments passed into `func`. kwargs : dict, optional A dictionary of keyword arguments passed into `func`. Returns ------- the return type of `func`. See Also -------- Series.pipe : Apply a function with arguments to a series. DataFrame.pipe: Apply a function with arguments to a dataframe. apply : Apply function to each group instead of to the full %(klass)s object. Notes ----- See more `here `_ Examples -------- %(examples)s a Call function producing a same-indexed %(klass)s on each group. Returns a %(klass)s having the same indexes as the original object filled with the transformed values. Parameters ---------- f : function, str Function to apply to each group. See the Notes section below for requirements. Accepted inputs are: - String - Python function - Numba JIT function with ``engine='numba'`` specified. Only passing a single function is supported with this engine. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. If a string is chosen, then it needs to be the name of the groupby method you want to use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or the global setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{'nopython': True, 'nogil': False, 'parallel': False}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs Keyword arguments to be passed into func. Returns ------- %(klass)s See Also -------- %(klass)s.groupby.apply : Apply function ``func`` group-wise and combine the results together. %(klass)s.groupby.aggregate : Aggregate using one or more operations over the specified axis. %(klass)s.transform : Call ``func`` on self producing a %(klass)s with the same axis shape as self. Notes ----- Each group is endowed the attribute 'name' in case you need to know which group you are working on. The current implementation imposes three requirements on f: * f must return a value that either has the same shape as the input subframe or can be broadcast to the shape of the input subframe. For example, if `f` returns a scalar it will be broadcast to have the same shape as the input subframe. * if this is a DataFrame, f must support application column-by-column in the subframe. If f also supports application to the entire subframe, then a fast path is used starting from the second chunk. * f must not mutate groups. Mutation is not supported and may produce unexpected results. See :ref:`gotchas.udf-mutation` for more details. When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. .. versionchanged:: 2.0.0 When using ``.transform`` on a grouped DataFrame and the transformation function returns a DataFrame, pandas now aligns the result's index with the input's index. You can call ``.to_numpy()`` on the result of the transformation function to avoid alignment. Examples -------- %(example)sa= Aggregate using one or more operations over the specified axis. Parameters ---------- func : function, str, list, dict or None Function to use for aggregating the data. If a function, must either work when passed a {klass} or when passed to {klass}.apply. Accepted combinations are: - function - string function name - list of functions and/or function names, e.g. ``[np.sum, 'mean']`` - dict of axis labels -> functions, function names or list of such. - None, in which case ``**kwargs`` are used with Named Aggregation. Here the output has one column for each element in ``**kwargs``. The name of the column is keyword, whereas the value determines the aggregation used to compute the values in the column. Can also accept a Numba JIT function with ``engine='numba'`` specified. Only passing a single function is supported with this engine. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs * If ``func`` is None, ``**kwargs`` are used to define the output names and aggregations via Named Aggregation. See ``func`` entry. * Otherwise, keyword arguments to be passed into func. Returns ------- {klass} See Also -------- {klass}.groupby.apply : Apply function func group-wise and combine the results together. {klass}.groupby.transform : Transforms the Series on each group based on the given function. {klass}.aggregate : Aggregate using one or more operations over the specified axis. Notes ----- When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See :ref:`gotchas.udf-mutation` for more details. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. {examples}c@s6eZdZdZdddddZddZd d d d Zd S) GroupByPlotzE Class implementing the .plot attribute for groupby objects. GroupByNone)groupbyreturncCs ||_dSN)_groupby)selfr^rc?/tmp/pip-unpacked-wheel-nbcvw55c/pandas/core/groupby/groupby.py__init__GszGroupByPlot.__init__cs fdd}d|_|j|S)Ncs |jSr`)plotrbargskwargsrcrdfKszGroupByPlot.__call__..frf)__name__raapply)rbrirjrkrcrhrd__call__JszGroupByPlot.__call__strnamecsfdd}|S)Ncsfdd}j|S)Ncst|jSr`)getattrrfrg)rirjrqrcrdrkSsz0GroupByPlot.__getattr__..attr..f)rarm)rirjrkrqrbrhrdattrRsz%GroupByPlot.__getattr__..attrrc)rbrqrtrcrsrd __getattr__QszGroupByPlot.__getattr__N)rl __module__ __qualname____doc__rernrurcrcrcrdr[Asr[c @sLeZdZUejddddddddd d d h BZd ed<d ed<dZded<dZded<ded<eddddZ eddddZ ee ddddZ ee ddddZ ee dddd Zed!d"Zed#d$Zeed%d&Zed'dd(d)Zed*ed+d,eed-d.d/d0d1Zed8d2dd3d4Zed5dd6d7ZdS)9 BaseGroupByas_indexaxisdropna exclusionsgrouper group_keyskeyslevelobjobservedsortrops.BaseGrouperN_KeysArgType | NoneIndexLabel | Noneboolintr_cCs t|jSr`)lengroupsrgrcrcrd__len__yszBaseGroupBy.__len__rocCs t|Sr`)object__repr__rgrcrcrdr}szBaseGroupBy.__repr__zdict[Hashable, np.ndarray]cCs|jjS)z4 Dict {group name -> group labels}. )r~rrgrcrcrdrszBaseGroupBy.groupscCs|jjSr`)r~ngroupsrgrcrcrdrszBaseGroupBy.ngroupsz$dict[Hashable, npt.NDArray[np.intp]]cCs|jjS)z5 Dict {group name -> group indices}. )r~indicesrgrcrcrdrszBaseGroupBy.indicesc s ddt|dkrgStjdkr6ttj}nd}|d}t|trt|tsbd}t|t|t|kszfdd|DWStk r}zd}t||W5d}~XYnXfd d|Dfd d |D}n|fd d |D}fd d|DS)zd Safe get multiple indices, translate keys for datelike to underlying repr. cSs4t|tjrddSt|tjr(ddSddSdS)NcSst|Sr`)rkeyrcrcrdzABaseGroupBy._get_indices..get_converter..cSs t|jSr`)rZasm8rrcrcrdrrcSs|Sr`rcrrcrcrdrr) isinstancedatetimenpZ datetime64)srcrcrd get_converters   z/BaseGroupBy._get_indices..get_converterrNzsz,BaseGroupBy._get_indices..zHmust supply a same-length tuple to get_group with multiple grouping keyscsg|] }|qSrcrc)rr)rrcrdrsc3s&|]}tddt|DVqdS)css|]\}}||VqdSr`rc)rrknrcrcrd sz5BaseGroupBy._get_indices...N)tuplezipr) convertersrcrdrsz+BaseGroupBy._get_indices..c3s|]}|VqdSr`rcr) converterrcrdrscsg|]}j|gqSrc)rgetrrgrcrdrs)rrnextiterrr ValueErrorKeyError)rbnamesZ index_sampleZ name_samplemsgerrrc)rrrrbrd _get_indicess.   zBaseGroupBy._get_indicescCs||gdS)zQ Safe get index, translate keys for datelike to underlying repr. r)r)rbrqrcrcrd _get_indexszBaseGroupBy._get_indexcCs>t|jtr|jS|jdk r8t|jr2|j|jS|jS|jSr`)rrrR _selectionr/_obj_with_exclusionsrgrcrcrd _selected_objs    zBaseGroupBy._selected_objzset[str]cCs |jSr`)r_dir_additionsrgrcrcrdrszBaseGroupBy._dir_additionsr\a >>> df = pd.DataFrame({'A': 'a b a b'.split(), 'B': [1, 2, 3, 4]}) >>> df A B 0 a 1 1 b 2 2 a 3 3 b 4 To get the difference between each groups maximum and minimum value in one pass, you can do >>> df.groupby('A').pipe(lambda x: x.max() - x.min()) B A a 2 b 2)klassexamplesz/Callable[..., T] | tuple[Callable[..., T], str]r#)funcr_cOstj||f||Sr`)compipe)rbrrirjrcrcrdrszBaseGroupBy.pipeDataFrame | SeriescCs8|dkr|j}||}t|s(t||j||jdS)a Construct DataFrame from group with provided name. Parameters ---------- name : object The name of the group to get as a DataFrame. obj : DataFrame, default None The DataFrame to take the DataFrame out of. If it is None, the object groupby was called on will be used. Returns ------- same type as obj Nr{)rrrrZ_take_with_is_copyr{)rbrqrZindsrcrcrd get_group s  zBaseGroupBy.get_groupz#Iterator[tuple[Hashable, NDFrameT]]cCsB|j}|jj|j|jd}t|tr>t|dkr>dd|D}|S)z Groupby iterator. Returns ------- Generator yielding sequence of (name, subsetted object) for each group rcss|]\}}|f|fVqdSr`rc)rrgrouprcrcrdr2sz'BaseGroupBy.__iter__..)rr~ get_iteratorrr{rlistr)rbrresultrcrcrd__iter__$s zBaseGroupBy.__iter__)N)rlrvrwrBZ _hidden_attrs__annotations__rrrrrpropertyrrrrrr*rrr)rr(_pipe_templaterrrrcrcrcrdrydsh    2  ryOutputFrameOrSeries)boundc@s^eZdZUdZded<ded<edd d d d ddd ddddddd ddZddddZeddddZddddZ eddddd d!Z ed"d"d#d$d%Z ed&d'd#d(d)Z d*d&d#d+d,Z ed d d#d-d.Zedd&d/d0d1d2Zdd3ddd4d5d6Zed'd7d8d9Zd:d;d<d=d>Zedd?d@dAZedd?dBdCZeedDjdEedFdGd ddHdIZedd:dJdKddd dLdMdNZedddPdd:dQdRdSZdTdPd:dTdUdVdWZeddd:ddPdXdYdZZdddd[d\d]d^Zeddd_d`daZed d d#dbdcZedddeZ ed ddfdgdhdiZ!ee"d:ddjdkZ#edlddmdndoZ$ee%dpdee&d ddqdrdsZ'ee%dpdee&d ddqdtduZ(ee%dpdee&d ddvdwZ)ee%dpde%e&dxd ddd;dzd{d|Z*ed dd}d~dZ+ee%dpdee&ddPdd;ddddZ,ee%dpdee&ddPdd;ddddZ-edddddddJdddZ.eddPddddZ/ee%dpdee&dJdddZ0ee1e2dd dddddPdd;dddZ3ee1e2dd dddddPdddZ4ee1e2dd dOddddPdd;dddZ5ee1e2dd dOddddPdd;dddZ6edddPdddZ7edddPdddZ8ed'dddZ9e1e:j;dd dddZ;eddZee%dpdee&ddddZ?edddddZ@ee%dpddddZAee%dpddddZBee"e%dpde%e&dxddddZCdddd dddZDedddddŜddDŽZEee%dpddddȜddʄZFee%dpddddȜdd̄ZGee%dpde%e&dxd ddddd[d dϜddфZHee%dpdee&d!d d dҜddԄZIee%dpdee&d"d d dҜddքZJee%dpdee&d#d[dd dלddلZKee%dpdee&d$d[dd dלddۄZLed%d:dddddޜddZMee%dpdd&dPd dddZNee%dpdee&d'dPd[d dddZOee%dpdee&d(dPdd dddZPee%dpde%e&dxd)dPd dddZQee%dpde%e&dxd*dPd dddZRedd dddZSeeTjUdfd"dd/d"dddZVed+ddddddddZWdS(,r\a Class for grouping and aggregating relational data. See aggregate, transform, and apply functions on this object. It's easiest to use obj.groupby(...) to use GroupBy, but you can also do: :: grouped = groupby(obj, ...) Parameters ---------- obj : pandas object axis : int, default 0 level : int, default None Level of MultiIndex groupings : list of Grouping objects Most users should ignore this exclusions : array-like, optional List of columns to exclude name : str Most users should ignore this Returns ------- **Attributes** groups : dict {group name -> group labels} len(grouped) : int Number of groups Notes ----- After grouping, see aggregate, apply, and transform functions. Here are some other brief notes about usage. When grouping by multiple groups, the result index will be a MultiIndex (hierarchical) by default. Iteration produces (key, group) tuples, i.e. chunking the data by group. So you can write code like: :: grouped = obj.groupby(keys, axis=axis) for key, group in grouped: # do something with the data Function calls on GroupBy, if not specially implemented, "dispatch" to the grouped data. So if you group a DataFrame and wish to invoke the std() method on each group, you can simply do: :: df.groupby(mapper).std() rather than :: df.groupby(mapper).aggregate(np.std) You can pass arguments to these "wrapped" functions, too. See the online documentation for full exposition on these topics and much more rr~rrzNrTFrrrrops.BaseGrouper | Nonezfrozenset[Hashable] | Noner]) rrr{rr~r} selectionrzrrrr|r_c Cs||_t|tstt|||_|s6|dkr6td||_||_| |_ | |_ | |_ | |_ |dkrt ||||| | |j d\}}}||_|||_||_|rt|nt|_dS)Nrz$as_index=False only valid for axis=0)r{rrrr|)rrrEAssertionErrortyperrrzrrrrr|rIrZ_get_axis_numberr{r~ frozensetr}) rbrrr{rr~r}rrzrrrr|rcrcrdres4  zGroupBy.__init__ro)rtcCsH||jkrt||S||jkr(||Stdt|jd|ddS)N'z' object has no attribute ')Z_internal_names_setr__getattribute__rAttributeErrorrrl)rbrtrcrcrdrus   zGroupBy.__getattr__rpcstt|j|t}d|jkrNdddksDdtjkrN|j d<fdd}||_ |t j krx| |S|t jk}|j||j|| d}|jjr|r||}|S)z.curried) is_transformnot_indexed_same)rrrrinspect signature parametersrrZ no_defaultr{rlrFZplotting_methodsrmtransformation_kernels_python_apply_generalr~has_dropped_na_set_result_index_ordered)rbrqrirjsigrrrrcrrd _op_via_applys&         zGroupBy._op_via_applyzIterable[Series]rcCs t|dSr`r&rgrcrcrd_iterate_slicesszGroupBy._iterate_slices)rrcCsVddlm}|jrn|sn|jrL|jj}|jj}|jj}|||j|||dd}n t t t |} |||j| d}n|s |||jd}|j |j} |jr|jjd} | dk} | | } | jr|j|j| st| j} |j| \}}|j||jd}n|j| |jdd}n|||jd}|jjd kr0|jjn|j}t|trR|dk rR||_|S) Nr)concatF)r{rlevelsrr)r{rrr{copyr) Zpandas.core.reshape.concatrrrzr~ result_indexrrr{rrangerr _get_axisr| group_infoZhas_duplicatesaxesequalsr8Zunique1d_valuesindexZget_indexer_non_uniquetakereindexrndimrqrrrR)rbvaluesrrrrZ group_levelsZ group_namesrrZaxlabelsmasktargetindexer_rqrcrcrd_concat_objectssB     zGroupBy._concat_objectsr)rr_cCs|j|j}|jjr4|jjs4|j||jdd}|St|j}|j||jdd}|j |jd}|jjr|j t t ||jd}|j||jdd}|S)NFrr) rrr{r~Z is_monotonicrset_axisrM result_ilocs sort_indexrrOr)rbrZobj_axisZoriginal_positionsrcrcrdr)sz!GroupBy._set_result_index_orderedzSeries | DataFramerDcCsrt|tr|}|j}tt|jjt|jtdd|jj DD]$\}}}|rH||krH| d||qH|S)NcSsg|] }|jqSrc)in_axis)rgrprcrcrdrMsz2GroupBy._insert_inaxis_grouper..r) rrRto_framecolumnsrreversedr~rZget_group_levels groupingsinsert)rbrrrqlevrrcrcrd_insert_inaxis_grouperCs    zGroupBy._insert_inaxis_grouperz"Mapping[base.OutputKey, ArrayLike]cCs t|dSr`rrbrrcrcrd_indexed_output_to_ndframeVsz"GroupBy._indexed_output_to_ndframecCs2|jdkr.|j}|j|jjr.|jj|_|S)Nr)r{r#rrrrrrcrcrd_maybe_transpose_result[s  zGroupBy._maybe_transpose_resultznpt.NDArray[np.float64] | None)rqscCsb|js*||}|}tt|jj}n|jj}|dk rDt||}||_ | |}|j ||dS)z Wraps the output of GroupBy aggregations into the expected result. Parameters ---------- result : Series, DataFrame Returns ------- Series or DataFrame Nr) rzrZ _consolidaterMrr~rr_insert_quantile_levelrr_reindex_output)rbrrrresrcrcrd_wrap_aggregated_outputfs   zGroupBy._wrap_aggregated_outputr)rrrcCs t|dSr`r)rbdatarrrrcrcrd_wrap_applied_outputszGroupBy._wrap_applied_output)r c Cs|jj\}}}t||}tj||dd}|j||jd}t|jj dkrTt d|j }t |t r||jj dj} || }||} t||\} } | | | |fS)NF)Z allow_fillrrzAMore than 1 grouping labels are not supported with engine='numba'r)r~rrSr8take_ndrr{to_numpyrrNotImplementedErrorrrrNrqZget_level_valuesrZgenerate_slices) rbr idsrr sorted_indexZ sorted_ids sorted_dataZ index_dataZ group_keyZsorted_index_datastartsendsrcrcrd _numba_preps&   zGroupBy._numba_preprzdict[str, bool] | None)r engine_kwargscGs|jstd|jdkr td|j}|jdkr4|n|}||\}}}} tj|ft |} | | ||df|} |j j } |jdkrd|j i} | } n d|ji} |j| fd| i| S) zp Perform groupby with a standard numerical aggregation function (e.g. mean) with Numba. z.fz6func must be a callable if args or kwargs are suppliedzmode.chained_assignment)rZis_builtin_funcrrohasattrrrcallabler TypeErrorrrrrr)rbrrirjrrkrrcr$rdrm#s0       *z GroupBy.applyrz bool | None)rkr rris_aggr_cCs2|j|||j\}}|dkr"|}|||||S)a Apply function f in python space Parameters ---------- f : callable Function to apply data : Series or DataFrame Data to apply f to not_indexed_same: bool, optional When specified, overrides the value of not_indexed_same. Apply behaves differently when the result index is equal to the input index, but this can be coincidental leading to value-dependent behavior. is_transform : bool, default False Indicator for whether the function is actually a transform and should not have group keys prepended. is_agg : bool, default False Indicator for whether the function is an aggregation. When the result is empty, we don't want to warn for this case. See _GroupBy._python_agg_general. Returns ------- Series or DataFrame data after applying f N)r~rmr{r )rbrkr rrr(rZmutatedrcrcrdrWs#zGroupBy._python_apply_generalrr numeric_only min_countaliasnpfunccCs"|j||||d}|j|jddS)Nhowaltr*r+r^method_cython_agg_general __finalize__r)rbr*r+r,r-rrcrcrd _agg_generals zGroupBy._agg_generalr)rrr0r_cCs|dk s t|jdkr$t|dd}n.t|j}|jddks@t|jdddf}|jj||dd}t |t rt |j ||j d}n|j tkr|jtdd}t||d S) zn Fallback to pure-python aggregation if _cython_operation raises NotImplementedError. NrFrrT)Zpreserve_dtypedtype)r)rrrRrDr#shapeilocr~Z agg_seriesrr=rZ_from_sequencer9rastyperQ)rbrrr0Zserr res_valuesrcrcrd_agg_py_fallbacks     zGroupBy._agg_py_fallbackr.c  sdj|ddddfdd }|}|}|} jdkr`| jdd} | S) Nr*rqrrr_csTz(jjd|fjdd}Wn&tk rNj|jd}YnX|S)N aggregaterr{r+)rr0)r~_cython_operationrrr>)rrr0r r/rjr+rbrcrd array_funcs z/GroupBy._cython_agg_general..array_funcrFr7)_get_data_to_aggregategrouped_reduce_wrap_agged_managerr r{Z infer_objects) rbr/r0r*r+rjrEnew_mgrroutrcrDrdr4s      zGroupBy._cython_agg_generalr)r/r*r{cKs t|dSr`r)rbr/r*r{rjrcrcrd_cython_transformszGroupBy._cython_transform)enginerc Ost|r"|j|f|d|i|St|p.|}t|tsL|j|f||S|tjkrld|d}t |nr|tj ks|tj krt ||||St |dd0t |ddt ||||}W5QRXW5QRX||SdS)Nrrz2' is not a valid function name for transform(name)rTrz)rUrrZget_cython_funcrroZ_transform_generalrFZtransform_kernel_allowlistrZcythonized_kernelsrrr temp_setattr_wrap_transform_fast_result)rbrrLrrirjrrrcrcrd _transforms,     $zGroupBy._transformcCs|j}|jj\}}}|j|jj|jdd}|jjdkrZt |j |}|j ||j |j d}n:|jdkrhdn|j}|j||dd}|j||j|d}|S)z7 Fast transform path for aggregations. Frr)rrqr)r{Zconvert_indicesr)rr~rrrr{rrr8r rrrrqZ_takerr)rbrrrrrJoutputr{rcrcrdrNs z#GroupBy._wrap_transform_fast_resultcCst|dkrtjgdd}ntt|}|rD|jj||jd}n^tjt|jj t d}| dd|| t <t|t|jjdddgj}|j|}|S)Nrint64r8rFTr)rrarrayr concatenaterrr{emptyrrfillr<rtilerr:r#where)rbrr|filteredrrcrcrd _apply_filter/s  $ zGroupBy._apply_filter np.ndarray) ascendingr_c Cs2|jj\}}}t||}||t|}}|dkrBtjdtjdStjd|dd|ddkf}ttjt |d|f}| } |r| t | ||8} n&t | tj|dddf|| } |jj rt |dktj| jtjdd} n| jtjdd} tj|tjd} tj|tjd| |<| | S) a. Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Notes ----- this is currently implementing sort=False (though the default is sort=True) for groupby in general rr8TNrrFr7)r~rrSrrrTrQZr_diffZnonzerocumsumrepeatrrWnanr<float64intpZarange) rbr[rrrZsortercountrunreprJrevrcrcrd_cumcount_array@s"  " &"zGroupBy._cumcount_arraycCs,t|jtr|jjSt|jts$t|jjSr`)rrrDZ_constructor_slicedrRrrrgrcrcrd_obj_1d_constructorhs zGroupBy._obj_1d_constructorzLiteral[('any', 'all')])val_testskipnac sLdddfdd }dddd dd d d }|jtjdttj|||d S)zO Shared func to call any / all Cython GroupBy implementations. rztuple[np.ndarray, type]valsr_cs^t|jr0r0t|}|r@|}d||<nt|tr@|j}|jt dd}| t j t fS)NTFr7) r3r9r6anyrrr;_datar<rviewrint8)rkrrircrd objs_to_boolws  z'GroupBy._bool_agg..objs_to_boolFrZrr)r inferencenullabler_cSs.|rt|jtdd|dkS|j|ddSdS)NFr7r)r<r<r)rrrrsrcrcrdresult_to_boolsz)GroupBy._bool_agg..result_to_bool)r* cython_dtypepre_processingpost_processingrhri)F)_get_cythonized_result libgroupbyZ group_any_allrr9ro)rbrhrirqrtrcrprd _bool_aggqs  zGroupBy._bool_aggr^rpcCs |d|S)a Return True if any value in the group is truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if any element is True within its respective group, False otherwise. rlrzrbrircrcrdrlsz GroupBy.anycCs |d|S)a Return True if all values in the group are truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if all elements are True within its respective group, False otherwise. r"r{r|rcrcrdr"sz GroupBy.allc s|}|jj\}dk|jdkdddfdd }||}||}t|dd||}W5QRX|j |d d S) z Compute count of group, excluding missing values. Returns ------- Series or DataFrame Count of values within each group. rrr)bvaluesr_csp|jdkr"t|dd@}nt|@}tj|d}rl|jdksRt|jddksdt|dS|S)Nrr)rZmax_binrr)rr6reshaperZcount_level_2drr:)r}ZmaskedZcountedrZ is_seriesrrrcrdhfuncs zGroupBy.count..hfuncrTr fill_value) rFr~rrrGrHrrMr r)rbr rrrInew_objrrcrrdrbs    z GroupBy.count)Zsee_alsocython)r*rLrcsLt|r ddlm}|||S|jdfddd}|j|jddSd S) aW Compute mean of groups, excluding missing values. Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. .. versionchanged:: 2.0.0 numeric_only no longer accepts ``None`` and defaults to ``False``. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 Returns ------- pandas.Series or pandas.DataFrame %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5], ... 'C': [1, 2, 1, 1, 2]}, columns=['A', 'B', 'C']) Groupby one column and return the mean of the remaining columns in each group. >>> df.groupby('A').mean() B C A 1 3.0 1.333333 2 4.0 1.500000 Groupby two columns and return the mean of the remaining column. >>> df.groupby(['A', 'B']).mean() C A B 1 2.0 2.0 4.0 1.0 2 3.0 1.0 5.0 2.0 Groupby one column and return the mean of only particular column in the group. >>> df.groupby('A')['B'].mean() A 1 3.0 2 4.0 Name: B, dtype: float64 r) sliding_meanmeancst|jdSNr*)rRrrrrcrdrArzGroupBy.mean..r0r*r^r1N)rUpandas.core._numba.kernelsrrr4r5r)rbr*rLrrrrcrrdrsM   z GroupBy.meanrcs(|jdfddd}|j|jddS)a Compute median of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. .. versionchanged:: 2.0.0 numeric_only no longer accepts ``None`` and defaults to False. Returns ------- Series or DataFrame Median of values within each group. mediancst|jdSr)rRrrrrcrdr]rz GroupBy.median..rr^r1r3)rbr*rrcrrdrFs  zGroupBy.medianrz str | None)ddofrLrr*c Csnt|r(ddlm}t||||Sdd}dddd d d }|jtjt tj |d |||d d}|SdS)a Compute standard deviation of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 numeric_only : bool, default False Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 .. versionchanged:: 2.0.0 numeric_only now defaults to ``False``. Returns ------- Series or DataFrame Standard deviation of values within each group. r sliding_varcSst|tr|jdfS|dfSr`)rr;rmrrcrcrd_preprocessings  z#GroupBy.std.._preprocessingFNrr)rsr_cSs@|r6|jdkr|dddf}tt||tjSt|S)Nrr)rr@rsqrtrnbool_)rkrrrs result_maskrcrcrd_postprocessings  z$GroupBy.std.._postprocessingTstd)rur* needs_countsrvrwrr/)FN) rUrrrrrrxryZ group_varr9r`) rbrrLrr*rrrrrcrcrdrbs$3    z GroupBy.stdcs@t|r"ddlm}|||S|jdfdd|dSdS)a Compute variance of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 numeric_only : bool, default False Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 .. versionchanged:: 2.0.0 numeric_only now defaults to ``False``. Returns ------- Series or DataFrame Variance of values within each group. rrvarcst|jdS)Nr)rRrrrrcrdrrzGroupBy.var..)r0r*rN)rUrrrr4)rbrrLrr*rrcrrdrs3  z GroupBy.varzSequence[Hashable] | None)subset normalizerr[r|r_c s|jdkrtd|rdnd}|j}|jdd|jjDttr`j}|krXgng} nxt j } |dk rt |t @} | rt d| d | } | rt d| d n| fd d t j D} t |jj} | D]0}t|||j|jd |d\}}}| t |j7} q|j| |j|j|jd}tt|}||_tdd| Drdd | D}tj|dd | Dd\}}|j|dd}|rt tt|jj|jj}|j|j||j|jdd}||}| d}|rtt|jj}|j!|dj"|d d}|j#r$|}n^|j}t$%|j&}||krPt d|d||_|'tt||_|(}||g|_ |}|j)|jddS) z Shared implementation of value_counts for SeriesGroupBy and DataFrameGroupBy. SeriesGroupBy additionally supports a bins argument. See the docstring of DataFrameGroupBy.value_counts for a description of arguments. rz1DataFrameGroupBy.value_counts only handles axis=0Z proportionrbcSsh|]}|jr|jqSrcrrqrgroupingrcrcrd sz(GroupBy._value_counts..NzKeys z0 in subset cannot be in the groupby column keys.z) in subset do not exist in the DataFrame.cs2g|]*\}}|kr|krjdd|fqSr`)r;)ridx_nameZ in_axis_namesrZ subsettedrcrdr%sz)GroupBy._value_counts..F)rr{rrr|)rrr|css&|]}t|jttfo|j VqdSr`)rgrouping_vectorr=rLZ _observedrrcrcrdrDsz(GroupBy._value_counts..cSsg|] }|jqSrc)rrpingrcrcrdrIscSsg|] }|jqSrcrprrcrcrdrKsrrr)rr|sumgr[)rZsort_remainingzColumn label 'z' is duplicate of result columnZ value_countsr1)*r{rrrr~rrrRrqsetrr enumeraterrIrr^rr|rsizerlrN from_productZ sortlevelrrrrZnlevelsZ droplevelZ transformZfillna sort_valuesrrzrZfill_missing_namesrZ set_names reset_indexr5)rbrrrr[r|rqrrrZ unique_colsZclashingZ doesnt_existrrr~rgbZ result_series levels_listZ multi_indexrZindexed_group_sizeZ index_levelrrrZ result_framercrrd _value_countss                 zGroupBy._value_countsrr*cCs|r>|jjdkr>t|jjs>tt|jd|d|jj|j||d}|jdkrj|t | }n`|j |j }| }|j |}|j |}|jdd|ft |jdd|f<|S)as Compute standard error of the mean of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. numeric_only : bool, default False Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 .. versionchanged:: 2.0.0 numeric_only now defaults to ``False``. Returns ------- Series or DataFrame Standard error of the mean of values within each group. rz.sem called with numeric_only=z and dtype rN)rrr2r9r'rrlrrrrbr differencer}uniqueZget_indexer_forr;)rbrr*rcolscountsrZ count_ilocsrcrcrdsemws   .z GroupBy.semc Cst|j}t|jtr*|j||jjd}n ||}t|dd|j |dd}W5QRX|j sp| d }|S)z Compute group sizes. Returns ------- DataFrame or Series Number of rows in each group as a Series if as_index is True or a DataFrame if as_index is False. rprzTrrr) r~rrrrRrgrqrrMrrzrenamerrrcrcrdrs   z GroupBy.sizer)fnamenoZmc)r*r+rLrc Cs`t|r ddlm}|||St|dd|j||dtjd}W5QRX|j |ddSdS)Nr) sliding_sumrTrr)r) rUrrrrrMr6rrr)rbr*r+rLrrrrcrcrdrs  z GroupBy.sumprod)r*r+cCs|j||dtjdS)Nrr))r6rr)rbr*r+rcrcrdrs z GroupBy.prodmincCs:t|r"ddlm}|||dS|j||dtjdSdS)Nrsliding_min_maxFrr))rUrrrr6rrrbr*r+rLrrrcrcrdrs  z GroupBy.minmaxcCs:t|r"ddlm}|||dS|j||dtjdSdS)NrrTrr))rUrrrr6rrrrcrcrdrs  z GroupBy.maxcCs$d ddddd}|j||d|dS) a Compute the first non-null entry of each column. Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. min_count : int, default -1 The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame First non-null of values within each group. See Also -------- DataFrame.groupby : Apply a function groupby to each row or column of a DataFrame. pandas.core.groupby.DataFrameGroupBy.last : Compute the last non-null entry of each column. pandas.core.groupby.DataFrameGroupBy.nth : Take the nth row from each group. Examples -------- >>> df = pd.DataFrame(dict(A=[1, 1, 3], B=[None, 5, 6], C=[1, 2, 3], ... D=['3/11/2000', '3/12/2000', '3/13/2000'])) >>> df['D'] = pd.to_datetime(df['D']) >>> df.groupby("A").first() B C D A 1 5.0 1 2000-03-11 3 6.0 3 2000-03-13 >>> df.groupby("A").first(min_count=2) B C D A 1 NaN 1.0 2000-03-11 3 NaN NaN NaT >>> df.groupby("A").first(numeric_only=True) B C A 1 5.0 1 3 6.0 3 rrrrr{cSsHdddd}t|tr&|j||dSt|tr8||Stt|dS)NrRrcSs&|jt|j}t|stjS|dS)z-Helper function for first item that isn't NA.rrRr7rrr_rZarrrcrcrdfirstB sz2GroupBy.first..first_compat..firstrrrDrmrRr'r)rr{rrcrcrd first_compatA s   z#GroupBy.first..first_compatrr))rr6)rbr*r+rrcrcrdr s0z GroupBy.firstcCs$d ddddd}|j||d|dS) az Compute the last non-null entry of each column. Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. min_count : int, default -1 The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame Last non-null of values within each group. See Also -------- DataFrame.groupby : Apply a function groupby to each row or column of a DataFrame. pandas.core.groupby.DataFrameGroupBy.first : Compute the first non-null entry of each column. pandas.core.groupby.DataFrameGroupBy.nth : Take the nth row from each group. Examples -------- >>> df = pd.DataFrame(dict(A=[1, 1, 3], B=[5, None, 6], C=[1, 2, 3])) >>> df.groupby("A").last() B C A 1 5.0 2 3 6.0 3 rrrrcSsHdddd}t|tr&|j||dSt|tr8||Stt|dS)NrRrcSs&|jt|j}t|stjS|dS)z,Helper function for last item that isn't NA.rrrrcrcrdlast} sz/GroupBy.last..last_compat..lastrr)rr{rrcrcrd last_compat| s   z!GroupBy.last..last_compatrr))rr)rbr*r+rrcrcrdrW s%z GroupBy.lastcCs|jjdkrl|j}t|j}|s(td|jjd|jdddd}dd d d g}|jj ||jj |d }| |S| d d|j }|js||}tt||_|S)a Compute open, high, low and close values of a group, excluding missing values. For multiple groupings, the result index will be a MultiIndex Returns ------- DataFrame Open, high, low and close values within each group. rzNo numeric types to aggregaterAohlcrrrBopenhighlowclose)rrcSs|Sr`)rrrcrcrdr rzGroupBy.ohlc..)rrrr2r9r'r~rCrZ_constructor_expanddimrrZ_apply_to_column_groupbysrrzrrPrr)rbrZ is_numericr=Z agg_namesrrcrcrdr s6     z GroupBy.ohlcc s|j}t|dkrN|jd}|jdkr2|}n|}|jjddSt |dd"|j fdd|dd}W5QRX|j dkr|jS|}|j s| |}tt||_|S) Nr percentilesincludeexcluderrzTcs|jdS)Nr)describerrrrrcrdr sz"GroupBy.describe..)r)rrrrZunstackrr#r;rrMrr{rzrrPr)rbrrrrZ describedrrcrrdr s0    zGroupBy.describecOsddlm}|||f||S)a< Provide resampling when using a TimeGrouper. Given a grouper, the function resamples it according to a string "string" -> "frequency". See the :ref:`frequency aliases ` documentation for more details. Parameters ---------- rule : str or DateOffset The offset string or object representing target grouper conversion. *args, **kwargs Possible arguments are `how`, `fill_method`, `limit`, `kind` and `on`, and other arguments of `TimeGrouper`. Returns ------- Grouper Return a new grouper with our resampler appended. See Also -------- Grouper : Specify a frequency to resample with when grouping by a key. DatetimeIndex.resample : Frequency conversion and resampling of time series. Examples -------- >>> idx = pd.date_range('1/1/2000', periods=4, freq='T') >>> df = pd.DataFrame(data=4 * [range(2)], ... index=idx, ... columns=['a', 'b']) >>> df.iloc[2, 0] = 5 >>> df a b 2000-01-01 00:00:00 0 1 2000-01-01 00:01:00 0 1 2000-01-01 00:02:00 5 1 2000-01-01 00:03:00 0 1 Downsample the DataFrame into 3 minute bins and sum the values of the timestamps falling into a bin. >>> df.groupby('a').resample('3T').sum() a b a 0 2000-01-01 00:00:00 0 2 2000-01-01 00:03:00 0 1 5 2000-01-01 00:00:00 5 1 Upsample the series into 30 second bins. >>> df.groupby('a').resample('30S').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:00:30 0 0 2000-01-01 00:01:00 0 1 2000-01-01 00:01:30 0 0 2000-01-01 00:02:00 0 0 2000-01-01 00:02:30 0 0 2000-01-01 00:03:00 0 1 5 2000-01-01 00:02:00 5 1 Resample by month. Values are assigned to the month of the period. >>> df.groupby('a').resample('M').sum() a b a 0 2000-01-31 0 3 5 2000-01-31 5 1 Downsample the series into 3 minute bins as above, but close the right side of the bin interval. >>> df.groupby('a').resample('3T', closed='right').sum() a b a 0 1999-12-31 23:57:00 0 1 2000-01-01 00:00:00 0 2 5 2000-01-01 00:00:00 5 1 Downsample the series into 3 minute bins and close the right side of the bin interval, but label each bin using the right edge instead of the left. >>> df.groupby('a').resample('3T', closed='right', label='right').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:03:00 0 2 5 2000-01-01 00:03:00 5 1 r)get_resampler_for_grouping)Zpandas.core.resampler)rbZrulerirjrrcrcrdresample sb zGroupBy.resamplerXcOs,ddlm}||jf||j|jd|S)a Return a rolling grouper, providing rolling functionality per group. Parameters ---------- window : int, timedelta, str, offset, or BaseIndexer subclass Size of the moving window. If an integer, the fixed number of observations used for each window. If a timedelta, str, or offset, the time period of each window. Each window will be a variable sized based on the observations included in the time-period. This is only valid for datetimelike indexes. To learn more about the offsets & frequency strings, please see `this link `__. If a BaseIndexer subclass, the window boundaries based on the defined ``get_window_bounds`` method. Additional rolling keyword arguments, namely ``min_periods``, ``center``, ``closed`` and ``step`` will be passed to ``get_window_bounds``. min_periods : int, default None Minimum number of observations in window required to have a value; otherwise, result is ``np.nan``. For a window that is specified by an offset, ``min_periods`` will default to 1. For a window that is specified by an integer, ``min_periods`` will default to the size of the window. center : bool, default False If False, set the window labels as the right edge of the window index. If True, set the window labels as the center of the window index. win_type : str, default None If ``None``, all points are evenly weighted. If a string, it must be a valid `scipy.signal window function `__. Certain Scipy window types require additional parameters to be passed in the aggregation function. The additional parameters must match the keywords specified in the Scipy window type method signature. on : str, optional For a DataFrame, a column label or Index level on which to calculate the rolling window, rather than the DataFrame's index. Provided integer column is ignored and excluded from result since an integer index is not used to calculate the rolling window. axis : int or str, default 0 If ``0`` or ``'index'``, roll across the rows. If ``1`` or ``'columns'``, roll across the columns. For `Series` this parameter is unused and defaults to 0. closed : str, default None If ``'right'``, the first point in the window is excluded from calculations. If ``'left'``, the last point in the window is excluded from calculations. If ``'both'``, the no points in the window are excluded from calculations. If ``'neither'``, the first and last points in the window are excluded from calculations. Default ``None`` (``'right'``). method : str {'single', 'table'}, default 'single' Execute the rolling operation per single column or row (``'single'``) or over the entire object (``'table'``). This argument is only implemented when specifying ``engine='numba'`` in the method call. Returns ------- RollingGroupby Return a new grouper with our rolling appended. See Also -------- Series.rolling : Calling object with Series data. DataFrame.rolling : Calling object with DataFrames. Series.groupby : Apply a function groupby to a Series. DataFrame.groupby : Apply a function groupby. Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 2], ... 'B': [1, 2, 3, 4], ... 'C': [0.362, 0.227, 1.267, -0.562]}) >>> df A B C 0 1 1 0.362 1 1 2 0.227 2 2 3 1.267 3 2 4 -0.562 >>> df.groupby('A').rolling(2).sum() B C A 1 0 NaN NaN 1 3.0 0.589 2 2 NaN NaN 3 7.0 0.705 >>> df.groupby('A').rolling(2, min_periods=1).sum() B C A 1 0 1.0 0.362 1 3.0 0.589 2 2 3.0 1.267 3 7.0 0.705 >>> df.groupby('A').rolling(2, on='B').sum() B C A 1 0 1 NaN 1 2 0.589 2 2 3 NaN 3 4 0.705 r)rX)_grouperZ _as_index)pandas.core.windowrXrr~rz)rbrirjrXrcrcrdrollingD s zGroupBy.rollingrVcOs(ddlm}||jf|d|ji|S)zc Return an expanding grouper, providing expanding functionality per group. r)rVr)rrVrr~)rbrirjrVrcrcrd expanding s zGroupBy.expandingrWcOs(ddlm}||jf|d|ji|S)zO Return an ewm grouper, providing ewm functionality per group. r)rWr)rrWrr~)rbrirjrWrcrcrdewm s z GroupBy.ewmzLiteral[('ffill', 'bfill')]) directionc s|dkr d}jj\}}}tj|ddjtjdd}|dkrJ|ddd}ttj||||j dd d d fd d } }| |} |} j d kr| j} jj| _jj| _| S)a  Shared function for `pad` and `backfill` to call Cython method. Parameters ---------- direction : {'ffill', 'bfill'} Direction passed to underlying Cython function. `bfill` will cause values to be filled backwards. `ffill` and any other values will default to a forward fill limit : int, default None Maximum number of consecutive values to fill. If `None`, this method will convert to -1 prior to passing to Cython Returns ------- `Series` or `DataFrame` with filled values See Also -------- pad : Returns Series with minimum number of char in object. backfill : Backward fill the missing values in the dataset. NrZ mergesort)kindFr7bfill)r sorted_labelsrlimitr|rr@cst|}|jdkr.blk_funcr)r~rrrr<rarryZgroup_fillna_indexerr|rFrmrHr{r#rrr) rbrrrrrrmgrres_mgrrrcrrd_fill s.      z GroupBy._fillcCs|jd|dS)a: Forward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.ffill: Returns Series with minimum number of char in object. DataFrame.ffill: Object with missing values filled or None if inplace=True. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. ffillrrrbrrcrcrdrG sz GroupBy.ffillcCs|jd|dS)a/ Backward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.bfill : Backward fill the missing values in the dataset. DataFrame.bfill: Backward fill the missing values in the dataset. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. rrrrrcrcrdr` sz GroupBy.bfillrKcCst|S)a Take the nth row from each group if n is an int, otherwise a subset of rows. Can be either a call or an index. dropna is not available with index notation. Index notation accepts a comma separated list of integers and slices. If dropna, will take the nth non-null row, dropna is either 'all' or 'any'; this is equivalent to calling dropna(how=dropna) before the groupby. Parameters ---------- n : int, slice or list of ints and slices A single nth value for the row or a list of nth values or slices. .. versionchanged:: 1.4.0 Added slice and lists containing slices. Added index notation. dropna : {'any', 'all', None}, default None Apply the specified dropna operation before counting which row is the nth row. Only supported if n is an int. Returns ------- Series or DataFrame N-th value within each group. %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5]}, columns=['A', 'B']) >>> g = df.groupby('A') >>> g.nth(0) A B 0 1 NaN 2 2 3.0 >>> g.nth(1) A B 1 1 2.0 4 2 5.0 >>> g.nth(-1) A B 3 1 4.0 4 2 5.0 >>> g.nth([0, 1]) A B 0 1 NaN 1 1 2.0 2 2 3.0 4 2 5.0 >>> g.nth(slice(None, -1)) A B 0 1 NaN 1 1 2.0 2 2 3.0 Index notation may also be used >>> g.nth[0, 1] A B 0 1 NaN 1 1 2.0 2 2 3.0 4 2 5.0 >>> g.nth[:-1] A B 0 1 NaN 1 1 2.0 2 2 3.0 Specifying `dropna` allows ignoring ``NaN`` values >>> g.nth(0, dropna='any') A B 1 1 2.0 2 2 3.0 When the specified ``n`` is larger than any of the groups, an empty DataFrame is returned >>> g.nth(3, dropna='any') Empty DataFrame Columns: [A, B] Index: [] )rKrgrcrcrdnthy s\z GroupBy.nthzPositionalIndexer | tuplezLiteral[('any', 'all', None)])rr|r_c Cs|s6||}|jj\}}}||dk@}||}|St|sFtd|dkr^td|dtt|}|jj ||j d}|j dkr|j dkr|jj }|jj ||j} |jjr| dk} t| t| } t| dd} n t||j |j |j |jd \} }}|j| |j|j|j d } | |S) Nrz4dropna option only supported for an integer argument)rlr"z_For a DataFrame or Series groupby.nth, dropna must be either None, 'any' or 'all', (was passed z).)r/r{ZInt64r8)rr{rr)rzrr{)"_make_mask_from_positional_indexerr~r_mask_selected_objr0rrrrr|r{rrZ codes_infoisinrrrrWrrMrIrr^rzr) rbrr|rrrrJZdroppedr{r~ZnullsrZgrbrcrcrd_nth sF      z GroupBy._nth?linearzfloat | AnyArrayLike)q interpolationr*c sddddddddddd fd d t|}|r<|g}tj|tjd }|jj\}}t|ttj ||d t|dkr| dnd}t |dk||dddfdd } |j |dd} | | } || } |r|| S|j| |dS)a Return group values at the given quantile, a la numpy.percentile. Parameters ---------- q : float or array-like, default 0.5 (50% quantile) Value(s) between 0 and 1 providing the quantile(s) to compute. interpolation : {'linear', 'lower', 'higher', 'midpoint', 'nearest'} Method to use when the desired quantile falls between two points. numeric_only : bool, default False Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 .. versionchanged:: 2.0.0 numeric_only now defaults to ``False``. Returns ------- Series or DataFrame Return type determined by caller of GroupBy object. See Also -------- Series.quantile : Similar method for Series. DataFrame.quantile : Similar method for DataFrame. numpy.percentile : NumPy method to compute qth percentile. Examples -------- >>> df = pd.DataFrame([ ... ['a', 1], ['a', 2], ['a', 3], ... ['b', 1], ['b', 3], ['b', 5] ... ], columns=['key', 'val']) >>> df.groupby('key').quantile() val key a 2.0 b 3.0 rz"tuple[np.ndarray, DtypeObj | None]rjcSst|rtdd}t|tr@t|jr@|jttj d}|j}nt |jrxt|t rf|jttj d}n|}ttj }nxt |jrt|t r|jttj d}nRt|jr|j}||fSt|t rt|rttj}|jttj d}n t|}||fS)Nz7'quantile' cannot be performed against 'object' dtypes!)r9Zna_value)r3r'rr;r2r9r floatrr_r1r?rQr-r5r.r`Zasarray)rkrrrJrcrcrd pre_processorH s.     z'GroupBy.quantile..pre_processorrZzDtypeObj | Noneznp.ndarray | None)rkrrr orig_valsr_cs|rt|trL|dk stdkr4t|s4t||St|||j|SnPt|r\dkst |r|d |j j }| |St|tj st||S|S)N>rmidpointi8)rr;rr.r@rr<Z numpy_dtyper1r5rnZ_ndarrayr9Z_from_backing_datar)rkrrrr)rrcrdpost_processork s6      z(GroupBy.quantile..post_processorr8)rrrrrrr@c sb|}t|tr*|j}tjftjd}n t|}d}t|j}|\}}d}|j dkrz|j d}t |t f}n}tj |ftjd} ||f} t| jtjdd} |r|dtj}|j dkr| d||| |dn.t|D]$} | | || || | | d q|j dkrD| d } |dk rT|d }n| |} | |||S) Nr8rrrFr7r)rr sort_indexerr)rrrK)rr;Z_maskrzerosrr6r5r9rr:Z broadcast_torrTr`Zlexsortr<rarnrrr~) rrrris_datetimelikerkrrncolsZ shaped_labelsrJorderZsort_arrr)rlabels_for_lexsortrnqsrrrcrdr sJ        "    z"GroupBy.quantile..blk_funcquantiler?r)r4rrRr`r~rrrryZgroup_quantilerrWrFrGrHr ) rbrrr*Z orig_scalarrrrZna_label_for_sortingrr rrrc)rrrrrrrrdr s.1#/6   zGroupBy.quantilercCs|j}||j}|jjd}|jjrBt|dktj|}tj }ntj }t dd|jj Drnt |ddd}|j|||d}|s|jd|}|S) a Number each group from 0 to the number of groups - 1. This is the enumerative complement of cumcount. Note that the numbers given to the groups match the order in which the groups would be seen when iterating over the groupby object, not the order they are first observed. Groups with missing keys (where `pd.isna()` is True) will be labeled with `NaN` and will be skipped from the count. Parameters ---------- ascending : bool, default True If False, number in reverse, from number of group - 1 to 0. Returns ------- Series Unique numbers for each group. See Also -------- .cumcount : Number the rows in each group. Examples -------- >>> df = pd.DataFrame({"color": ["red", None, "red", "blue", "blue", "red"]}) >>> df color 0 red 1 None 2 red 3 blue 4 blue 5 red >>> df.groupby("color").ngroup() 0 1.0 1 NaN 2 1.0 3 0.0 4 0.0 5 1.0 dtype: float64 >>> df.groupby("color", dropna=False).ngroup() 0 1 1 2 2 1 3 0 4 0 5 1 dtype: int64 >>> df.groupby("color", dropna=False).ngroup(ascending=False) 0 1 1 0 2 1 3 2 4 2 5 1 dtype: int64 rrcss|] }|jVqdSr`)Z_passed_categoricalrrcrcrdr; sz!GroupBy.ngroup..Zdense) ties_methodrr8)rrr{r~rrrrWr_r`rQrlrrrgr)rbr[rrZcomp_idsr9rrcrcrdngroup s@  zGroupBy.ngroupcCs&|j|j}|j|d}|||S)a Number each item in each group from 0 to the length of that group - 1. Essentially this is equivalent to .. code-block:: python self.apply(lambda x: pd.Series(np.arange(len(x)), x.index)) Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Returns ------- Series Sequence number of each element within each group. See Also -------- .ngroup : Number the groups themselves. Examples -------- >>> df = pd.DataFrame([['a'], ['a'], ['a'], ['b'], ['b'], ['a']], ... columns=['A']) >>> df A 0 a 1 a 2 a 3 b 4 b 5 a >>> df.groupby('A').cumcount() 0 0 1 1 2 2 3 0 4 1 5 3 dtype: int64 >>> df.groupby('A').cumcount(ascending=False) 0 3 1 2 2 1 3 1 4 0 5 0 dtype: int64 r)rrr{rfrg)rbr[rZ cumcountsrcrcrdcumcountD s7 zGroupBy.cumcountaveragekeep)r2r[ na_optionpctr{r_c sr|dkrd}t|||||ddkr\dd<fdd}|j||jd d }|S|jdd d S)a_ Provide the rank of values within each group. Parameters ---------- method : {'average', 'min', 'max', 'first', 'dense'}, default 'average' * average: average rank of group. * min: lowest rank in group. * max: highest rank in group. * first: ranks assigned in order they appear in the array. * dense: like 'min', but rank always increases by 1 between groups. ascending : bool, default True False for ranks by high (1) to low (N). na_option : {'keep', 'top', 'bottom'}, default 'keep' * keep: leave NA values where they are. * top: smallest rank if ascending. * bottom: smallest rank if descending. pct : bool, default False Compute percentage rank of data within each group. axis : int, default 0 The axis of the object over which to compute the rank. Returns ------- DataFrame with ranking of values within each group %(see_also)s Examples -------- >>> df = pd.DataFrame( ... { ... "group": ["a", "a", "a", "a", "a", "b", "b", "b", "b", "b"], ... "value": [2, 4, 2, 3, 5, 1, 2, 4, 1, 5], ... } ... ) >>> df group value 0 a 2 1 a 4 2 a 2 3 a 3 4 a 5 5 b 1 6 b 2 7 b 4 8 b 1 9 b 5 >>> for method in ['average', 'min', 'max', 'dense', 'first']: ... df[f'{method}_rank'] = df.groupby('group')['value'].rank(method) >>> df group value average_rank min_rank max_rank dense_rank first_rank 0 a 2 1.5 1.0 2.0 1.0 1.0 1 a 4 4.0 4.0 4.0 3.0 4.0 2 a 2 1.5 1.0 2.0 1.0 2.0 3 a 3 3.0 3.0 3.0 2.0 3.0 4 a 5 5.0 5.0 5.0 4.0 5.0 5 b 1 1.5 1.0 2.0 1.0 1.0 6 b 2 3.0 3.0 3.0 2.0 3.0 7 b 4 4.0 4.0 4.0 3.0 4.0 8 b 1 1.5 1.0 2.0 1.0 2.0 9 b 5 5.0 5.0 5.0 4.0 5.0 >topbottomrz3na_option must be one of 'keep', 'top', or 'bottom')rr[r r rrr2cs|jfddS)NF)r{r*)rankrr{rjrcrdr rzGroupBy.rank..Trr F)r*r{)r )rpoprrrK) rbr2r[r r r{rrkrrcrrdr  s2Hz GroupBy.rank)r{r_csHtd|ddgdkr<fdd}|j||jddS|jd S) zq Cumulative product for each group. Returns ------- Series or DataFrame cumprodr*rircs|jfdiSNr{)rrrrcrdr rz!GroupBy.cumprod..Tr)rnvZvalidate_groupby_funcrrrKrbr{rirjrkrcrrdr s zGroupBy.cumprodcsHtd|ddgdkr<fdd}|j||jddS|jd S) zm Cumulative sum for each group. Returns ------- Series or DataFrame r]r*rircs|jfdiSr)r]rrrcrdrrz GroupBy.cumsum..Tr)r]rrrcrrdr] s zGroupBy.cumsum)r{r*r_c sR|dd}dkrBfdd}|j}|r2|}|j||ddS|jd||dS) zm Cumulative min for each group. Returns ------- Series or DataFrame riTrcstj|Sr`)rZminimum accumulaterrrcrdrrz GroupBy.cummin..rcumminr*rirrZ_get_numeric_datarrKrbr{r*rjrirkrrcrrdrs  zGroupBy.cumminc sR|dd}dkrBfdd}|j}|r2|}|j||ddS|jd||dS) zm Cumulative max for each group. Returns ------- Series or DataFrame riTrcstj|Sr`)rmaximumrrrrcrdr-rz GroupBy.cummax..rcummaxrrrrcrrdrs  zGroupBy.cummaxany_allznp.dtype) base_funcrur*rr/c srtstdr(ts(td|j} | j\} } t| ddddfdd } |j|d} | | }||}||S) a Get result for Cythonized functions. Parameters ---------- base_func : callable, Cythonized function to be called cython_dtype : np.dtype Type of the array that will be modified by the Cython call. numeric_only : bool, default False Whether only numeric datatypes should be computed needs_counts : bool, default False Whether the counts should be a part of the Cython call pre_processing : function, default None Function to be applied to `values` prior to passing to Cython. Function should return a tuple where the first element is the values to be passed to Cython and the second element is an optional type which the values should be converted to after being returned by the Cython operation. This function is also responsible for raising a TypeError if the values have an invalid type. Raises if `needs_values` is False. post_processing : function, default None Function to be applied to result of Cython function. Should accept an array of values as the first argument and type inferences as its second argument, i.e. the signature should be (ndarray, Type). If `needs_nullable=True`, a third argument should be `nullable`, to allow for processing specific to nullable values. how : str, default any_all Determines if any/all cython interface or std interface is used. **kwargs : dict Extra arguments to be passed back to Cython funcs Returns ------- `Series` or `DataFrame` with filled values z%'post_processing' must be a callable!z$'pre_processing' must be a callable!)rrr@c sl|j}|jdkrdn|jd}tj|d}||f}t|d}d}rntjtjd}t||d}|jj dk}|}|rdkr| d}r|\}}|j dd }|jdkr|d }t||d }dkst |t rt| tj}|jdkr|d d}t||d }dkr.blk_funcr?) r&rr~rrrFrGrHr )rbrrur*rrvrwr/rjr~rrrrrrJrcr&rdrx7s/    "H  zGroupBy._get_cythonized_resultperiodsr{c sdk sdkr4fdd}|j||jddS|jj\}}}tjt|tjd} t | |||j } | j |j | j |j | fidd} | S) a Shift each group by periods observations. If freq is passed, the index will be increased using the periods and the freq. Parameters ---------- periods : int, default 1 Number of periods to shift. freq : str, optional Frequency string. axis : axis to shift, default 0 Shift direction. fill_value : optional The scalar value to use for newly introduced missing values. Returns ------- Series or DataFrame Object shifted within each group. See Also -------- Index.shift : Shift values of Index. Nrcs|Sr`)shiftrr{rfreqr(rcrdrrzGroupBy.shift..Trr8)rZ allow_dups)rrr~rrrrrQryZgroup_shift_indexerrZ_reindex_with_indexersr{r) rbr(r+r{rrkrrrZ res_indexerrrrcr*rdr)sz GroupBy.shift)r(r{r_csdkr|fddS|j}|jd}ddg|jdkrX|jkr|d}n4fd d |jD}t|r|d d |D}||S) a First discrete difference of element. Calculates the difference of each element compared with another element in the group (default is element in previous row). Parameters ---------- periods : int, default 1 Periods to shift for calculating difference, accepts negative values. axis : axis to shift, default 0 Take difference over rows (0) or columns (1). Returns ------- Series or DataFrame First differences. rcs|jdS)Nr')r\r)r{r(rcrdrrzGroupBy.diff..r'roZint16rfloat32csg|]\}}|kr|qSrcrc)rcr9) dtypes_to_f32rcrdrsz GroupBy.diff..cSsi|] }|dqS)r,rc)rr-rcrcrd sz GroupBy.diff..) rmrr)rr9r<Zdtypesitemsr)rbr(r{rshiftedZ to_coercerc)r{r.r(rdr\s   z GroupBy.diffrr)r( fill_methodr{c sdk sdkr6fdd}|j||jddSdkrFddt|d}|j|jj|j|jd }|j|jd } || d S) z Calculate pct_change of each value to previous entry in group. Returns ------- Series or DataFrame Percentage changes within each group. Nrcs|jdS)N)r(r2rr+r{) pct_changerr{r2r+rr(rcrdr-s z$GroupBy.pct_change..Trrr)r{r)r(r+r{r) rrrrr^r~codesr{rr)) rbr(r2rr+r{rkZfilledZfill_grpr1rcr4rdr3szGroupBy.pct_change)rr_cCs|td|}||S)a Return first n rows of each group. Similar to ``.apply(lambda x: x.head(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Parameters ---------- n : int If positive: number of entries to include from start of each group. If negative: number of entries to exclude from end of each group. Returns ------- Series or DataFrame Subset of original Series or DataFrame as determined by n. %(see_also)s Examples -------- >>> df = pd.DataFrame([[1, 2], [1, 4], [5, 6]], ... columns=['A', 'B']) >>> df.groupby('A').head(1) A B 0 1 2 2 5 6 >>> df.groupby('A').head(-1) A B 0 1 2 Nrslicerrbrrrcrcrdhead@s#z GroupBy.headcCs,|r|t| d}n |g}||S)a Return last n rows of each group. Similar to ``.apply(lambda x: x.tail(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Parameters ---------- n : int If positive: number of entries to include from end of each group. If negative: number of entries to exclude from start of each group. Returns ------- Series or DataFrame Subset of original Series or DataFrame as determined by n. %(see_also)s Examples -------- >>> df = pd.DataFrame([['a', 1], ['a', 2], ['b', 1], ['b', 2]], ... columns=['A', 'B']) >>> df.groupby('A').tail(1) A B 1 a 2 3 b 2 >>> df.groupby('A').tail(-1) A B 1 a 2 3 b 2 Nr7r9rcrcrdtailfs$ z GroupBy.tailznpt.NDArray[np.bool_])rr_cCsD|jjd}||dk@}|jdkr,|j|S|jjdd|fSdS)a Return _selected_obj with mask applied to the correct axis. Parameters ---------- mask : np.ndarray[bool] Boolean mask to apply. Returns ------- Series or DataFrame Filtered _selected_obj. rrN)r~rr{rr;)rbrrrcrcrdrs     zGroupBy._mask_selected_objr")rPrrr_c Cs2|jj}t|dkr|S|jr"|Stdd|Ds8|Sdd|D}|jj}|dk rj|||dg}tj||d}|j r| }|j r|j |j|dd d |i}|jf|Std dt|D} t| d krt| \} } |jt| dd }||jjj|d |d}t| d kr&|j| d}|jddS)aI If we have categorical groupers, then we might want to make sure that we have a fully re-indexed output to the levels. This means expanding the output space to accommodate all values in the cartesian product of our groups, regardless of whether they were observed in the data or not. This will expand the output space if there are missing groups. The method returns early without modifying the input if the number of groupings is less than 2, self.observed == True or none of the groupers are categorical. Parameters ---------- output : Series or DataFrame Object resulting from grouping and applying an operation. fill_value : scalar, default np.NaN Value to use for unobserved categories if self.observed is False. qs : np.ndarray[float64] or None, default None quantile values, only relevant for quantile. Returns ------- Series or DataFrame Object (potentially) re-indexed to include all possible groups. rcss|]}t|jttfVqdSr`)rrr=rLrrcrcrdrsz*GroupBy._reindex_output..cSsg|] }|jqSrc)Z group_indexrrcrcrdrsz+GroupBy._reindex_output..NrrFrcss"|]\}}|jr||jfVqdSr`r)rrrrcrcrdrsr)rr{)rr)rT)drop)r~rrrrlrappendrNrrrrzrZ_get_axis_namer{rrrrr<Z set_indexrr) rbrPrrrrrrdZ in_axis_grpsZg_numsZg_namesrcrcrdrsP         zGroupBy._reindex_outputz int | Nonez float | NonezSequence | Series | NonezRandomState | None)rfracreplaceweights random_statecCs|jjr|jSt|||}|dk r8tj|j||jd}t|}|j |j|j}g} |D]r\} } |j | } t | } |dk r|}n|dk st t || }tj| |||dkrdn|| |d}| | |q\t| } |jj| |jdS)a Return a random sample of items from each group. You can use `random_state` for reproducibility. .. versionadded:: 1.1.0 Parameters ---------- n : int, optional Number of items to return for each group. Cannot be used with `frac` and must be no larger than the smallest group unless `replace` is True. Default is one if `frac` is None. frac : float, optional Fraction of items to return. Cannot be used with `n`. replace : bool, default False Allow or disallow sampling of the same row more than once. weights : list-like, optional Default None results in equal probability weighting. If passed a list-like then values must have the same length as the underlying DataFrame or Series object and will be used as sampling probabilities after normalization within each group. Values must be non-negative with at least one positive element within each group. random_state : int, array-like, BitGenerator, np.random.RandomState, np.random.Generator, optional If int, array-like, or BitGenerator, seed for random number generator. If np.random.RandomState or np.random.Generator, use as given. .. versionchanged:: 1.4.0 np.random.Generator objects now accepted Returns ------- Series or DataFrame A new object of same type as caller containing items randomly sampled within each group from the caller object. See Also -------- DataFrame.sample: Generate random samples from a DataFrame object. numpy.random.choice: Generate a random sample from a given 1-D numpy array. Examples -------- >>> df = pd.DataFrame( ... {"a": ["red"] * 2 + ["blue"] * 2 + ["black"] * 2, "b": range(6)} ... ) >>> df a b 0 red 0 1 red 1 2 blue 2 3 blue 3 4 black 4 5 black 5 Select one row at random for each distinct value in column a. The `random_state` argument can be used to guarantee reproducibility: >>> df.groupby("a").sample(n=1, random_state=1) a b 4 black 4 2 blue 2 1 red 1 Set `frac` to sample fixed proportions rather than counts: >>> df.groupby("a")["b"].sample(frac=0.5, random_state=2) 5 5 2 2 0 0 Name: b, dtype: int64 Control sample probabilities within groups by setting weights: >>> df.groupby("a").sample( ... n=1, ... weights=[1, 1, 1, 0, 0, 1], ... random_state=1, ... ) a b 5 black 5 2 blue 2 0 red 0 Nr)rr@rArB)rrTr9Zprocess_sampling_sizeZpreprocess_weightsr{rrBr~rrrrroundr=rrSr)rbrr?r@rArBrZ weights_arrZgroup_iteratorZsampled_indicesrrZ grp_indicesZ group_sizeZ sample_sizeZ grp_samplercrcrdr9 s:`      zGroupBy.sample) NrNNNNTTTFT)FF)N)FF)NFF)Fr)Fr)Fr)T)T)T)FrN)F)rNNF)rNNF)NFTFT)rF)FrNN)Fr)FrNN)FrNN)Fr)Fr)NNN)N)N)N)N)rrF)T)T)rTrFr)r)r)rF)rF)FFNNr)rNrN)rr)rrNNr)r6)r6)NNFNN)Xrlrvrwrxrrrerurrrrrrrr r rrrrr( _apply_docsformatrmrr6r>r4rKrOrNrYrfrrgrzr)_common_see_alsorlr"rbrrrrrrrr+_groupby_agg_method_templaterrrrrrrrDrrrrrrrrrrrrrr rr]rrrxr)r\r3r:r;rrNaNrr9rcrcrcrdr\:s2 C*0 (= - !#'/-,(# '&*VP<+E:%% e  U]@YR9 _ ,&&#(ar\TrErrrr)rbyr{r~rr_cCsXt|trddlm}|}n*t|tr8ddlm}|}ntd|||||||dS)Nr) SeriesGroupBy)DataFrameGroupByzinvalid type: )rrr{r~r)rrRZpandas.core.groupby.genericrJrDrKr')rrIr{r~rrJrrKrcrcrd get_groupbys    rLrMznpt.NDArray[np.float64]rN)rrr_cst||jrvtt|}t|\}}t|j|g}fdd|jDt |t|g}t|||j dgd}nt ||g}|S)a Insert the sequence 'qs' of quantiles as the inner-most level of a MultiIndex. The quantile level in the MultiIndex is a repeated copy of 'qs'. Parameters ---------- idx : Index qs : np.ndarray[float64] Returns ------- MultiIndex csg|]}t|qSrc)rr^)rrrrcrdrsz*_insert_quantile_level..N)rr5r) rZ _is_multirrNrMZ factorizerrr5rrVrr)rrZ lev_codesrrr5mircrMrdrs &r)NrNT)rx __future__rr functoolsrrrtextwraprtypingrrrr r r r r rrrrrr#ZnumpyrZpandas._config.configrZ pandas._libsrrZpandas._libs.algosrZpandas._libs.groupbyZ_libsr^ryZpandas._libs.missingrZpandas._typingrrrrrrrrr r!r"r#r$Zpandas.compat.numpyr%rZ pandas.errorsr&r'Zpandas.util._decoratorsr(r)r*r+Zpandas.core.dtypes.castr,Zpandas.core.dtypes.commonr-r.r/r0r1r2r3r4r5Zpandas.core.dtypes.missingr6r7Z pandas.corer8r9Zpandas.core._numbar:Zpandas.core.arraysr;r<r=r>r?r@rAZpandas.core.baserBrCZpandas.core.commoncorecommonrZpandas.core.framerDZpandas.core.genericrEZpandas.core.groupbyrFrGrHZpandas.core.groupby.grouperrIZpandas.core.groupby.indexingrJrKZpandas.core.indexes.apirLrMrNrOrPZpandas.core.internals.blocksrQZpandas.core.seriesrRZpandas.core.sortingrSZpandas.core.util.numba_rTrUrrVrWrXrFrDrGrZ_transform_templateZ _agg_templater[Z _KeysArgTyperyrr\rLrrcrcrcrds  <   <  ,  $        5A-3bS   T n