Mattis landed in the Afghan capital on Wednesday, alongside NATO chief Jens Stoltenberg, and was met with several rockets landing in and around the airport, local police reported.
There were no reports of casualties or damage after the attack, which was claimed by the Taliban.
The trip marks Mattis' first visit to the country since US President Donald Trump's pledge last month to win America's longest war.
His schedule will include a meeting with Afghan President Ashraf Ghani to discuss NATO's US-led "train and assist" mission, designed to strengthen Afghanistan's security forces.
"Discussions will focus on the NATO-Afghanistan partnership, including the ongoing NATO-led Resolute Support Mission in support of the Afghan National Defence and Security Forces," a statement on Resolute Support's Facebook page said on Wednesday.
Afghanistan has struggled to combat offensives by Taliban fighters in the country since the withdrawal of US-led NATO troops at the end of 2014.
US generals have labelled the war in Afghanistan, which will enter its 16th year in October, a" stalemate", despite the estimated $1 trillion the US has invested in fighting and reconstruction in the country.
The US is pressing NATO partners to increase their own troop levels in Afghanistan following US President Donald Trump's announcement in August that he was giving the Pentagon authority to ramp up the deployment of forces in the country.
Under Trump's plan, more than 3,000 troops will be added to the 11,000 US forces already on the ground in Afghanistan.
NATO allies have around 5,000 troops currently deployed around the country.
The Taliban have promised to turn Afghanistan into a "graveyard" for foreign forces and have frequently carried out deadly attacks in areas of the country they control.
As of February, about 60 percent of Afghanistan's 407 districts were reported to be under government control, according to the Special Inspector General for Afghanistan Reconstruction, a US watchdog agency.
Mattis arrived in Kabul following a visit to New Delhi, during which he met his Indian counterpart Nirmala Sitharaman who said India would not send troops to Afghanistan as part of Trump's plan.