IPsec encryption encrypts only the tunnel through which the data is flowing or it encrypts the underlying packets as well?

Since IPsec as a protocol suite is using Encapsulating Security Payloads (ESP), it protects the "whole inner IP packet", see: https://en.wikipedia.org/wiki/IPsec#Encapsulating_Security_Payload.

Excerpt:

 However, in Tunnel Mode, where the entire original IP packet is encapsulated with a new packet header added, ESP protection is afforded to the whole inner IP packet (including the inner header) while the outer header (including any outer IPv4 options or IPv6 extension headers) remains unprotected. ESP operates directly on top of IP, using IP protocol number 50.