"axes": "index, columns",

"klass": "DataFrame",

"axes_single_arg": "{0 or 'index', 1 or 'columns'}",

"axis": """axis : {0 or 'index', 1 or 'columns'}, default 0

If 0 or 'index': apply function to each column.

If 1 or 'columns': apply function to each row.""",

"inplace": """

inplace : bool, default False

Whether to modify the DataFrame rather than creating a new one.""",

"optional_by": """

by : str or list of str

Name or list of names to sort by.

- if `axis` is 0 or `'index'` then `by` may contain index

levels and/or column labels.

- if `axis` is 1 or `'columns'` then `by` may contain column

levels and/or index labels.""",

"optional_reindex": """

labels : array-like, optional

New labels / index to conform the axis specified by 'axis' to.

index : array-like, optional

New labels for the index. Preferably an Index object to avoid

duplicating data.

columns : array-like, optional

New labels for the columns. Preferably an Index object to avoid

duplicating data.

axis : int or str, optional

Axis to target. Can be either the axis name ('index', 'columns')

or number (0, 1).""",

Merge DataFrame or named Series objects with a database-style join.

A named Series object is treated as a DataFrame with a single named column.

The join is done on columns or indexes. If joining columns on

columns, the DataFrame indexes *will be ignored*. Otherwise if joining indexes

on indexes or indexes on a column or columns, the index will be passed on.

When performing a cross merge, no column specifications to merge on are

allowed.

.. warning::

If both key columns contain rows where the key is a null value, those

rows will be matched against each other. This is different from usual SQL

join behaviour and can lead to unexpected results.

Parameters

----------%s

right : DataFrame or named Series

Object to merge with.

how : {'left', 'right', 'outer', 'inner', 'cross'}, default 'inner'

Type of merge to be performed.

* left: use only keys from left frame, similar to a SQL left outer join;

preserve key order.

* right: use only keys from right frame, similar to a SQL right outer join;

preserve key order.

* outer: use union of keys from both frames, similar to a SQL full outer

join; sort keys lexicographically.

* inner: use intersection of keys from both frames, similar to a SQL inner

join; preserve the order of the left keys.

* cross: creates the cartesian product from both frames, preserves the order

of the left keys.

on : label or list

Column or index level names to join on. These must be found in both

DataFrames. If `on` is None and not merging on indexes then this defaults

to the intersection of the columns in both DataFrames.

left_on : label or list, or array-like

Column or index level names to join on in the left DataFrame. Can also

be an array or list of arrays of the length of the left DataFrame.

These arrays are treated as if they are columns.

right_on : label or list, or array-like

Column or index level names to join on in the right DataFrame. Can also

be an array or list of arrays of the length of the right DataFrame.

These arrays are treated as if they are columns.

left_index : bool, default False

Use the index from the left DataFrame as the join key(s). If it is a

MultiIndex, the number of keys in the other DataFrame (either the index

or a number of columns) must match the number of levels.

right_index : bool, default False

Use the index from the right DataFrame as the join key. Same caveats as

left_index.

sort : bool, default False

Sort the join keys lexicographically in the result DataFrame. If False,

the order of the join keys depends on the join type (how keyword).

suffixes : list-like, default is ("_x", "_y")

A length-2 sequence where each element is optionally a string

indicating the suffix to add to overlapping column names in

`left` and `right` respectively. Pass a value of `None` instead

of a string to indicate that the column name from `left` or

`right` should be left as-is, with no suffix. At least one of the

values must not be None.

copy : bool, default False

If False, avoid copy if possible.

.. note::

The `copy` keyword will change behavior in pandas 3.0.

`Copy-on-Write

<https://pandas.pydata.org/docs/dev/user_guide/copy_on_write.html>`__

will be enabled by default, which means that all methods with a

`copy` keyword will use a lazy copy mechanism to defer the copy and

ignore the `copy` keyword. The `copy` keyword will be removed in a

future version of pandas.

You can already get the future behavior and improvements through

enabling copy on write ``pd.options.mode.copy_on_write = True``

.. deprecated:: 3.0.0

indicator : bool or str, default False

If True, adds a column to the output DataFrame called "_merge" with

information on the source of each row. The column can be given a different

name by providing a string argument. The column will have a Categorical

type with the value of "left_only" for observations whose merge key only

appears in the left DataFrame, "right_only" for observations

whose merge key only appears in the right DataFrame, and "both"

if the observation's merge key is found in both DataFrames.

validate : str, optional

If specified, checks if merge is of specified type.

* "one_to_one" or "1:1": check if merge keys are unique in both

left and right datasets.

* "one_to_many" or "1:m": check if merge keys are unique in left

dataset.

* "many_to_one" or "m:1": check if merge keys are unique in right

dataset.

* "many_to_many" or "m:m": allowed, but does not result in checks.

Returns

-------

DataFrame

A DataFrame of the two merged objects.

See Also

--------

merge_ordered : Merge with optional filling/interpolation.

merge_asof : Merge on nearest keys.

DataFrame.join : Similar method using indices.

Examples

--------

>>> df1 = pd.DataFrame({'lkey': ['foo', 'bar', 'baz', 'foo'],

... 'value': [1, 2, 3, 5]})

>>> df2 = pd.DataFrame({'rkey': ['foo', 'bar', 'baz', 'foo'],

... 'value': [5, 6, 7, 8]})

>>> df1

lkey value

0 foo 1

1 bar 2

2 baz 3

3 foo 5

>>> df2

rkey value

0 foo 5

1 bar 6

2 baz 7

3 foo 8

Merge df1 and df2 on the lkey and rkey columns. The value columns have

the default suffixes, _x and _y, appended.

>>> df1.merge(df2, left_on='lkey', right_on='rkey')

lkey value_x rkey value_y

0 foo 1 foo 5

1 foo 1 foo 8

2 bar 2 bar 6

3 baz 3 baz 7

4 foo 5 foo 5

5 foo 5 foo 8

Merge DataFrames df1 and df2 with specified left and right suffixes

appended to any overlapping columns.

>>> df1.merge(df2, left_on='lkey', right_on='rkey',

... suffixes=('_left', '_right'))

lkey value_left rkey value_right

0 foo 1 foo 5

1 foo 1 foo 8

2 bar 2 bar 6

3 baz 3 baz 7

4 foo 5 foo 5

5 foo 5 foo 8

Merge DataFrames df1 and df2, but raise an exception if the DataFrames have

any overlapping columns.

>>> df1.merge(df2, left_on='lkey', right_on='rkey', suffixes=(False, False))

Traceback (most recent call last):

...

ValueError: columns overlap but no suffix specified:

Index(['value'], dtype='object')

>>> df1 = pd.DataFrame({'a': ['foo', 'bar'], 'b': [1, 2]})

>>> df2 = pd.DataFrame({'a': ['foo', 'baz'], 'c': [3, 4]})

>>> df1

a b

0 foo 1

1 bar 2

>>> df2

a c

0 foo 3

1 baz 4

>>> df1.merge(df2, how='inner', on='a')

a b c

0 foo 1 3

>>> df1.merge(df2, how='left', on='a')

a b c

0 foo 1 3.0

1 bar 2 NaN

>>> df1 = pd.DataFrame({'left': ['foo', 'bar']})

>>> df2 = pd.DataFrame({'right': [7, 8]})

>>> df1

left

0 foo

1 bar

>>> df2

right

0 7

1 8

>>> df1.merge(df2, how='cross')

left right

0 foo 7

1 foo 8

2 bar 7

3 bar 8

"""

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Data structure also contains labeled axes (rows and columns).

Arithmetic operations align on both row and column labels. Can be

thought of as a dict-like container for Series objects. The primary

pandas data structure.

Parameters

----------

data : ndarray (structured or homogeneous), Iterable, dict, or DataFrame

Dict can contain Series, arrays, constants, dataclass or list-like objects. If

data is a dict, column order follows insertion-order. If a dict contains Series

which have an index defined, it is aligned by its index. This alignment also

occurs if data is a Series or a DataFrame itself. Alignment is done on

Series/DataFrame inputs.

If data is a list of dicts, column order follows insertion-order.

index : Index or array-like

Index to use for resulting frame. Will default to RangeIndex if

no indexing information part of input data and no index provided.

columns : Index or array-like

Column labels to use for resulting frame when data does not have them,

defaulting to RangeIndex(0, 1, 2, ..., n). If data contains column labels,

will perform column selection instead.

dtype : dtype, default None

Data type to force. Only a single dtype is allowed. If None, infer.

copy : bool or None, default None

Copy data from inputs.

For dict data, the default of None behaves like ``copy=True``. For DataFrame

or 2d ndarray input, the default of None behaves like ``copy=False``.

If data is a dict containing one or more Series (possibly of different dtypes),

``copy=False`` will ensure that these inputs are not copied.

.. versionchanged:: 1.3.0

See Also

--------

DataFrame.from_records : Constructor from tuples, also record arrays.

DataFrame.from_dict : From dicts of Series, arrays, or dicts.

read_csv : Read a comma-separated values (csv) file into DataFrame.

read_table : Read general delimited file into DataFrame.

read_clipboard : Read text from clipboard into DataFrame.

Notes

-----

Please reference the :ref:`User Guide <basics.dataframe>` for more information.

Examples

--------

Constructing DataFrame from a dictionary.

>>> d = {"col1": [1, 2], "col2": [3, 4]}

>>> df = pd.DataFrame(data=d)

>>> df

col1 col2

0 1 3

1 2 4

Notice that the inferred dtype is int64.

>>> df.dtypes

col1 int64

col2 int64

dtype: object

To enforce a single dtype:

>>> df = pd.DataFrame(data=d, dtype=np.int8)

>>> df.dtypes

col1 int8

col2 int8

dtype: object

Constructing DataFrame from a dictionary including Series:

>>> d = {"col1": [0, 1, 2, 3], "col2": pd.Series([2, 3], index=[2, 3])}

>>> pd.DataFrame(data=d, index=[0, 1, 2, 3])

col1 col2

0 0 NaN

1 1 NaN

2 2 2.0

3 3 3.0

Constructing DataFrame from numpy ndarray:

>>> df2 = pd.DataFrame(

... np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=["a", "b", "c"]

... )

>>> df2

a b c

0 1 2 3

1 4 5 6

2 7 8 9

Constructing DataFrame from a numpy ndarray that has labeled columns:

>>> data = np.array(

... [(1, 2, 3), (4, 5, 6), (7, 8, 9)],

... dtype=[("a", "i4"), ("b", "i4"), ("c", "i4")],

... )

>>> df3 = pd.DataFrame(data, columns=["c", "a"])

>>> df3

c a

0 3 1

1 6 4

2 9 7

Constructing DataFrame from dataclass:

>>> from dataclasses import make_dataclass

>>> Point = make_dataclass("Point", [("x", int), ("y", int)])

>>> pd.DataFrame([Point(0, 0), Point(0, 3), Point(2, 3)])

x y

0 0 0

1 0 3

2 2 3

Constructing DataFrame from Series/DataFrame:

>>> ser = pd.Series([1, 2, 3], index=["a", "b", "c"])

>>> df = pd.DataFrame(data=ser, index=["a", "c"])

>>> df

0

a 1

c 3

>>> df1 = pd.DataFrame([1, 2, 3], index=["a", "b", "c"], columns=["x"])

>>> df2 = pd.DataFrame(data=df1, index=["a", "c"])

>>> df2

x

a 1

c 3

"""

_internal_names_set = {"columns", "index"} | NDFrame._internal_names_set

_typ = "dataframe"

_HANDLED_TYPES = (Series, Index, ExtensionArray, np.ndarray)

_accessors: set[str] = {"sparse"}

_hidden_attrs: frozenset[str] = NDFrame._hidden_attrs | frozenset([])

_mgr: BlockManager

# similar to __array_priority__, positions DataFrame before Series, Index,

# and ExtensionArray. Should NOT be overridden by subclasses.

__pandas_priority__ = 4000

@property

def _constructor(self) -> type[DataFrame]:

return DataFrame

def _constructor_from_mgr(self, mgr, axes) -> DataFrame:

df = DataFrame._from_mgr(mgr, axes=axes)

if type(self) is DataFrame:

# This would also work `if self._constructor is DataFrame`, but

# this check is slightly faster, benefiting the most-common case.

return df

elif type(self).__name__ == "GeoDataFrame":

# Shim until geopandas can override their _constructor_from_mgr

# bc they have different behavior for Managers than for DataFrames

return self._constructor(mgr)

# We assume that the subclass __init__ knows how to handle a

# pd.DataFrame object.

return self._constructor(df)

_constructor_sliced: Callable[..., Series] = Series

def _constructor_sliced_from_mgr(self, mgr, axes) -> Series:

ser = Series._from_mgr(mgr, axes)

ser._name = None # caller is responsible for setting real name

if type(self) is DataFrame:

# This would also work `if self._constructor_sliced is Series`, but

# this check is slightly faster, benefiting the most-common case.

return ser

# We assume that the subclass __init__ knows how to handle a

# pd.Series object.

return self._constructor_sliced(ser)

# ----------------------------------------------------------------------

# Constructors

def __init__(

self,

data=None,

index: Axes | None = None,

columns: Axes | None = None,

dtype: Dtype | None = None,

copy: bool | None = None,

) -> None:

allow_mgr = False

if dtype is not None:

dtype = self._validate_dtype(dtype)

if isinstance(data, DataFrame):

data = data._mgr

allow_mgr = True

if not copy:

# if not copying data, ensure to still return a shallow copy

# to avoid the result sharing the same Manager

data = data.copy(deep=False)

if isinstance(data, BlockManager):

if not allow_mgr:

# GH#52419

warnings.warn(

f"Passing a {type(data).__name__} to {type(self).__name__} "

"is deprecated and will raise in a future version. "

"Use public APIs instead.",

DeprecationWarning,

stacklevel=1, # bump to 2 once pyarrow 15.0 is released with fix

)

data = data.copy(deep=False)

# first check if a Manager is passed without any other arguments

# -> use fastpath (without checking Manager type)

if index is None and columns is None and dtype is None and not copy:

# GH#33357 fastpath

NDFrame.__init__(self, data)

return

is_pandas_object = isinstance(data, (Series, Index, ExtensionArray))

data_dtype = getattr(data, "dtype", None)

original_dtype = dtype

# GH47215

if isinstance(index, set):

raise ValueError("index cannot be a set")

if isinstance(columns, set):

raise ValueError("columns cannot be a set")

if copy is None:

if isinstance(data, dict):

# retain pre-GH#38939 default behavior

copy = True

elif not isinstance(data, (Index, DataFrame, Series)):

copy = True

else:

copy = False

if data is None:

index = index if index is not None else default_index(0)

columns = columns if columns is not None else default_index(0)

dtype = dtype if dtype is not None else pandas_dtype(object)

data = []

if isinstance(data, BlockManager):

mgr = self._init_mgr(

data, axes={"index": index, "columns": columns}, dtype=dtype, copy=copy

)

elif isinstance(data, dict):

# GH#38939 de facto copy defaults to False only in non-dict cases

mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy)

elif isinstance(data, ma.MaskedArray):

from numpy.ma import mrecords

# masked recarray

if isinstance(data, mrecords.MaskedRecords):

raise TypeError(

"MaskedRecords are not supported. Pass "

"{name: data[name] for name in data.dtype.names} "

"instead"

)

# a masked array

data = sanitize_masked_array(data)

mgr = ndarray_to_mgr(

data,

index,

columns,

dtype=dtype,

copy=copy,

)

elif isinstance(data, (np.ndarray, Series, Index, ExtensionArray)):

if data.dtype.names:

# i.e. numpy structured array

data = cast(np.ndarray, data)

mgr = rec_array_to_mgr(

data,

index,

columns,

dtype,

copy,

)

elif getattr(data, "name", None) is not None:

# i.e. Series/Index with non-None name

mgr = dict_to_mgr(

# error: Item "ndarray" of "Union[ndarray, Series, Index]" has no

# attribute "name"

{data.name: data}, # type: ignore[union-attr]

index,

columns,

dtype=dtype,

copy=copy,

)

else:

mgr = ndarray_to_mgr(

data,

index,

columns,

dtype=dtype,

copy=copy,

)

# For data is list-like, or Iterable (will consume into list)

elif is_list_like(data):

if not isinstance(data, abc.Sequence):

if hasattr(data, "__array__"):

# GH#44616 big perf improvement for e.g. pytorch tensor

data = np.asarray(data)

else:

data = list(data)

if len(data) > 0:

if is_dataclass(data[0]):

data = dataclasses_to_dicts(data)

if not isinstance(data, np.ndarray) and treat_as_nested(data):

# exclude ndarray as we may have cast it a few lines above

if columns is not None:

columns = ensure_index(columns)

arrays, columns, index = nested_data_to_arrays(

# error: Argument 3 to "nested_data_to_arrays" has incompatible

# type "Optional[Collection[Any]]"; expected "Optional[Index]"

data,

columns,

index, # type: ignore[arg-type]

dtype,

)

mgr = arrays_to_mgr(

arrays,

columns,

index,

dtype=dtype,

)

else:

mgr = ndarray_to_mgr(

data,

index,

columns,

dtype=dtype,

copy=copy,

)

else:

mgr = dict_to_mgr(

{},

index,

columns if columns is not None else default_index(0),

dtype=dtype,

)

# For data is scalar

else:

if index is None or columns is None:

raise ValueError("DataFrame constructor not properly called!")

index = ensure_index(index)

columns = ensure_index(columns)

if not dtype:

dtype, _ = infer_dtype_from_scalar(data)

# For data is a scalar extension dtype

if isinstance(dtype, ExtensionDtype):

# TODO(EA2D): special case not needed with 2D EAs

values = [

construct_1d_arraylike_from_scalar(data, len(index), dtype)

for _ in range(len(columns))

]

mgr = arrays_to_mgr(values, columns, index, dtype=None)

else:

arr2d = construct_2d_arraylike_from_scalar(

data,

len(index),

len(columns),

dtype,

copy,

)

mgr = ndarray_to_mgr(

arr2d,

index,

columns,

dtype=arr2d.dtype,

copy=False,

)

NDFrame.__init__(self, mgr)

if original_dtype is None and is_pandas_object and data_dtype == np.object_:

if self.dtypes.iloc[0] != data_dtype:

warnings.warn(

"Dtype inference on a pandas object "

"(Series, Index, ExtensionArray) is deprecated. The DataFrame "

"constructor will keep the original dtype in the future. "

"Call `infer_objects` on the result to get the old "

"behavior.",

FutureWarning,

stacklevel=2,

)

# ----------------------------------------------------------------------

def __dataframe__(

self, nan_as_null: bool = False, allow_copy: bool = True

) -> DataFrameXchg:

"""

Return the dataframe interchange object implementing the interchange protocol.

Parameters

----------

nan_as_null : bool, default False

`nan_as_null` is DEPRECATED and has no effect. Please avoid using

it; it will be removed in a future release.

allow_copy : bool, default True

Whether to allow memory copying when exporting. If set to False

it would cause non-zero-copy exports to fail.

Returns

-------

DataFrame interchange object

The object which consuming library can use to ingress the dataframe.

See Also

--------

DataFrame.from_records : Constructor from tuples, also record arrays.

DataFrame.from_dict : From dicts of Series, arrays, or dicts.

Notes

-----

Details on the interchange protocol:

https://data-apis.org/dataframe-protocol/latest/index.html

Examples

--------

>>> df_not_necessarily_pandas = pd.DataFrame({"A": [1, 2], "B": [3, 4]})

>>> interchange_object = df_not_necessarily_pandas.__dataframe__()

>>> interchange_object.column_names()

Index(['A', 'B'], dtype='object')

>>> df_pandas = pd.api.interchange.from_dataframe(

... interchange_object.select_columns_by_name(["A"])

... )

>>> df_pandas

A

0 1

1 2

These methods (``column_names``, ``select_columns_by_name``) should work

for any dataframe library which implements the interchange protocol.

"""

from pandas.core.interchange.dataframe import PandasDataFrameXchg

return PandasDataFrameXchg(self, allow_copy=allow_copy)

def __arrow_c_stream__(self, requested_schema=None):

"""

Export the pandas DataFrame as an Arrow C stream PyCapsule.

This relies on pyarrow to convert the pandas DataFrame to the Arrow

format (and follows the default behaviour of ``pyarrow.Table.from_pandas``

in its handling of the index, i.e. store the index as a column except

for RangeIndex).

This conversion is not necessarily zero-copy.

Parameters

----------

requested_schema : PyCapsule, default None

The schema to which the dataframe should be casted, passed as a

PyCapsule containing a C ArrowSchema representation of the

requested schema.

Returns

-------

PyCapsule

"""

pa = import_optional_dependency("pyarrow", min_version="14.0.0")

if requested_schema is not None:

requested_schema = pa.Schema._import_from_c_capsule(requested_schema)

table = pa.Table.from_pandas(self, schema=requested_schema)

return table.__arrow_c_stream__()

# ----------------------------------------------------------------------

@property

def axes(self) -> list[Index]:

"""

Return a list representing the axes of the DataFrame.

It has the row axis labels and column axis labels as the only members.

They are returned in that order.