Hands-On Time Series Analysis with Python Course: Manipulating Time Series Data [1/6]

A practical guide for time series data manipulation in Python Pandas

Time series data is one of the most common data types in the industry, and you will probably be working with it in your career. Therefore, understanding how to work with it and how to apply analytical and forecasting techniques is critical for every aspiring data scientist.

In this series of articles, I will go through the basic techniques to work with time-series data, starting from data manipulation, analysis, and visualization to understand your data and prepare it for, and then using statistical, machine, and deep learning techniques for forecasting and classification. It will be more of a practical guide in which I will be applying each discussed and explained concept to real data.

This series will consist of 6 articles:

1. Manipulating Time Series Data in Python Pandas [A Practical Guide] (You are here!)

2. Time Series Analysis in Python Pandas [A Practical Guide]

3. Visualizing Time Series Data in Python [A Practical Guide]

4. Time Series Forecasting with ARIMA Models In Python [Part 1]

5. Time Series Forecasting with ARIMA Models In Python [Part 2]

6. Machine Learning for Time Series Data [Regression]

Time Series Data/ Photo by Chris Liverani on Unsplash

Table of contents:

1. Working with Time Series in Pandas

2. Basic Time Series Metrics & Resampling

3. Window Functions: Rolling & Expanding Metrics

4. Building a value-weighted index

All the codes and data used can be found in this repository

---

---

1. Working with Time Series in Pandas

This section lays the foundations to leverage the powerful time-series functionality made available by how Pandas represents dates, in particular by the DateTimeIndex. You will learn how to create and manipulate date information and time series, and how to do calculations with time-aware DataFrames to shift your data in time or create period-specific returns.

1.1. How to use data and times with pandas

The basic building block of creating a time series data in Python using Pandas time stamp (pd.Timestamp) is shown in the example below:

import pandas as pd 
from datetime import datetime # To manually create dates
time_stamp = pd.Timestamp(datetime(2017, 1, 1))

The timestamp object has many attributes that can be used to retrieve specific time information of your data, such as year and weekday. In the example below, the year of the data is retrieved.

time_stamp.year

The second building block is the period object. The period object has a freq attribute to store the frequency information. The default is monthly frequency, and you can convert from one frequency to another as shown in the example below.

period = pd.Period('2017-01')
period

The output shows that the default frequency is monthly. You can convert it into a daily frequency using the code below.

period.asfreq('D') # convert to daily

You can also convert a period to a timestamp and vice versa. This is shown in the example below.

period.to_timestamp().to_period('M')

You can do basic data arithmetic operations, for example, starting with a period object for January 2017 at a monthly frequency, just add the number 2 to get a monthly period for March 2017. This is shown in the example below.

period = pd.Period('2017-01')
period+2

The output is shown in the image below:

To create a time series, you will need to create a sequence of dates. To create a sequence of Timestamps, use the pandas function date_range. You need to specify a start date, and/or end date, or many periods. The default is daily frequency. The function returns the sequence of dates as a DateTimeindex with frequency information. You will recognize the first element as a pandas Timestamp.

This is shown in the example below, and the output is shown in the figure below:

index = pd.date_range(start='2017-1-1', periods=12, freq='M')
index