Substitute striker Mohanad Ibrahim was the Syrian's hero in extra time with his two goals adding to those from Iyad Mando and Ahmed Al Omeir during regulation time.
Needing to overturn a 2-0 deficit, Mohammed Kwid's men turned on an attacking display in front of a large home crowd.
The crucial period before and after halftime again proved decisive with the home side scoring four minutes before the break through midfielder Mando and then three minutes after when striker Al Omeir landed an opportunists goal after teammate Fabio Santos had miscued a penalty.
Without Mexican striker Jared Borgetti the Saudis lacked a attacking presence and they could not find the crucial goal during the opening 90 minutes as the aggregate scores remained deadlocked.
Ibrahim entered the match on 96 minutes and quickly played a major part when he put Al Karama in front eight minutes later.
His second goal at 110th minute sparked wild celebrations around the stadium.
Al Karama manager Kwid was elated after his side's historic victory.
"I said after the first match in Jeddah that there is everything to play for in the second leg and that is what happened.
Al Karama's Iyad Mando (r) does
the defensive work as well
"We faced a big team that has won the tournament twice and they had a 2-0 advantage. Therefore, winning today is a great achievement," he said.
While Al Ittihad coach Vahid Halilhodzic was honest when counting the cost of his side's failure, rueing his player's arrogant attitude.
"Our defeat today is a disaster. We committed fatal mistakes in the game and Al Karama, who went into the game with an attacking mindset, took full advantage of it.
"Al Karama are a good team with quality players and they can adapt their game to the circumstance. They were playing at home tonight and their strong support helped them a lot.
"I warned my players before the match to take this game seriously because the tables could be turned on us and that is what happened. Unfortunately, some of the players went into the game with an arrogant attitude.
"I'm sad and upset as this is the biggest defeat that I have suffered during my coaching career," he lamented.
Al Karama had become the first side from Syria ever to reach the quarter finals and their record breaking run will continue when they face Kuwaiti side Al Qadsiya in the last four.
The Kuwaiti champions were also surprise victors when they disposed of 2003 champions Al Ain of the UAE with a thumping 3-0 away win to advance with a 5-2 aggregate victory.
The other semi final will be an all-Korean affair after South Korea's Chonbuk came from a goal down to beat 10-man Shanghai Shenhua 4-2.
A brace from Ze Carlo and strikes from Yeom Ki-lun and Chung Jung-kwan cancelled out Gao Lin's opener for the Chinese side as Chonbuk went through 4-3 on aggregate.
While the other Korean quarterfinalist Ulsan were never in danger of losing their six goal advantage as they coasted to a 1-0 win over Saudi Arabia’s Al Shabab to run out 7-0 winners overall.
The first leg of the semifinals will be held on September 27 with the return legs scheduled for October 18.