🐼Pandas

A library for data manipulation and analysis

🔗 GitHub | Website

pandas is a Python package that provides fast, flexible, and expressive data structures designed to make working with "relational" or "labeled" data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way towards this goal.

Pandas and Exploratory Data Analysis (EDA)

Pandas is a powerful Python library commonly used in Exploratory Data Analysis (EDA). It provides data structures and functions to efficiently manipulate and analyze structured data, making it a fundamental tool for data analysis tasks.

Key features of pandas in the context of EDA:

1. Data Structures: Pandas introduces two primary data structures, namely Series and DataFrame. A Series is a one-dimensional array-like object that can hold any data type, while a DataFrame is a two-dimensional table with labeled columns and rows, similar to a spreadsheet or a SQL table. These data structures allow for efficient storage and manipulation of data during EDA.

2. Data Loading: Pandas provides functions to read data from various file formats, including CSV, Excel, SQL databases, and more. The read_csv(), read_excel(), and read_sql() functions are commonly used to import data into pandas DataFrames.

3. Data Cleaning: EDA often involves cleaning and preprocessing the data. Pandas offers a wide range of functions to handle missing data, duplicate values, data type conversions, and other data cleaning tasks. Functions like dropna(), fillna(), drop_duplicates(), and astype() are commonly used for data cleaning.

4. Data Exploration: Pandas provides numerous functions for exploring and summarizing data. You can perform operations like sorting, filtering, grouping, and aggregating data using functions like sort_values(), query(), groupby(), and agg(). These operations allow you to gain insights into the structure and characteristics of your data.

5. Descriptive Statistics: Pandas offers a wide range of statistical functions to compute descriptive statistics of your data. You can calculate measures such as mean, median, standard deviation, quantiles, correlation, and more using functions like mean(), median(), std(), quantile(), and corr(). These statistics help in understanding the distribution and relationships within the dataset.

6. Data Visualization: While pandas itself does not provide visualization capabilities, it seamlessly integrates with other libraries like Matplotlib and Seaborn. You can directly plot visualizations from pandas DataFrames using functions like plot(), plot.bar(), plot.hist(), and more. This integration allows you to generate insightful plots and charts as part of your EDA process.

Overall, pandas simplifies and accelerates the data manipulation and analysis tasks involved in EDA. Its intuitive data structures and extensive functions enable data scientists and analysts to efficiently explore and understand their datasets, making it an essential tool in the EDA workflow.

Main Features

Here are just a few of the things that pandas does well:

• Easy handling of missing data (represented as NaN, NA, or NaT) in floating point as well as non-floating point data

• Size mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects

• Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can simply ignore the labels and let Series, DataFrame, etc. automatically align the data for you in computations

• Powerful, flexible group by functionality to perform split-apply-combine operations on data sets, for both aggregating and transforming data

• Make it easy to convert ragged, differently-indexed data in other Python and NumPy data structures into DataFrame objects

• Intelligent label-based slicing, fancy indexing, and subsetting of large data sets

• Intuitive merging and joining data sets

• Flexible reshaping and pivoting of data sets

• Hierarchical labeling of axes (possible to have multiple labels per tick)

• Robust IO tools for loading data from flat files (CSV and delimited), Excel files, databases, and saving/loading data from the ultrafast HDF5 format

• Time series-specific functionality: date range generation and frequency conversion, moving window statistics, date shifting and lagging

Where to get it

The source code is currently hosted on GitHub at: https://github.com/pandas-dev/pandas

Binary installers for the latest released version are available at the Python Package Index (PyPI) and on Conda.

# conda
conda install -c conda-forge pandas

# or PyPI
pip install pandas

The list of changes to pandas between each release can be found here. For full details, see the commit logs at https://github.com/pandas-dev/pandas.

Dependencies

• NumPy - Adds support for large, multi-dimensional arrays, matrices and high-level mathematical functions to operate on these arrays
• python-dateutil - Provides powerful extensions to the standard datetime module
• pytz - Brings the Olson tz database into Python which allows accurate and cross platform timezone calculations

See the full installation instructions for minimum supported versions of required, recommended and optional dependencies.

Installation from sources

To install pandas from source you need Cython in addition to the normal dependencies above. Cython can be installed from PyPI:

pip install cython

In the pandas directory (same one where you found this file after cloning the git repo), execute:

python setup.py install

or for installing in development mode:

python -m pip install -e . --no-build-isolation --no-use-pep517

or alternatively

python setup.py develop

See the full instructions for installing from source.

Documentation

The official documentation is hosted on PyData.org.