The humiliating loss meant Madrid were eliminated 5-3 on aggregate in the round of 16, leaving the club trophy-less.
Madrid: Kings of Europe no more. Real Madrid's Champions League reign is over.
More than 1,000 days since winning the first of three consecutive European titles, Real Madrid unprecedented era of dominance ended with a whimper when the Spanish giant was stunned 4-1 by Ajax at Santiago Bernabeu Stadium on Tuesday.
The humiliating loss meant Madrid were eliminated 5-3 on aggregate in the round of 16, leaving the club looking at a trophy-less season in its first campaign since mega-star Cristiano Ronaldo left for Juventus.
"We were not going to win the Champions League forever," said Real Madrid defender Nacho Fernandez, who was sent off in the final minutes of the game.
"Reigns eventually come to an end. Something like this was going to happen sooner or later."
In the end, the club renowned for assembling its team with expensive "Galactico" signings was undone by a club centered around a core of homegrown young talents.
Ajax, which is enjoying a resurgence thanks to young stars like Frenkie de Jong and Matthijs de Ligt, needed less than 20 minutes to reverse a 2-1 deficit from the first leg, with Hakim Ziyech scoring in the seventh minute and David Neres in the 18th.
Dusan Tadic, who set up both first-half goals, added to the lead in the 62nd, and Lasse Schone sealed the Dutch team's triumph with a free kick from a tight angle in the 72nd. Marco Asensio had pulled the hosts closer a couple of minutes earlier.
"We always believed we had a chance, and we had 4,000 fans here who believed as well," Ajax coach Erick ten Hag said.
"Over the two matches we definitely deserve to progress to the next round." In Tuesday's other game, Tottenham won 1-0 at Borussia Dortmund to go through 4-0 on aggregate, reaching the quarterfinals for the first time since 2011.
Ajax was playing in the last 16 for the first time since 2006 and will be making its first quarterfinal appearance since 2003.
For Madrid, it was the fourth straight home loss in all competitions, something that hadn't happened since 2004, and capped one of the worst weeks in the club's recent history.
The team was coming off two demoralising home losses to arch-rival Barcelona 1-0 in the Spanish league and 3-0 in the semifinals of the Copa del Rey.
Out of the Copa del Rey and 12 points behind Barcelona in the Spanish league, the Champions League was Real Madrid's only realistic chance of winning anything this season.
"We are sad. We are hurt," Real Madrid coach Santiago Solari said.
"We gave our best but it wasn't enough. All we can do is come back tomorrow and keep working hard. The season continues and we have to keep playing."
Madrid hadn't lost a Champions League knockout tie since the 2015 semifinal against Juventus. It had reached at least the semifinals in eight straight seasons and hadn't failed to make it past the last 16 since 2010.
In the group stage this season, it had lost 3-0 to CSKA Moscow at the Bernabeu for what had been its worst ever European loss.
Madrid was without suspended captain Sergio Ramos, who was sanctioned by UEFA for deliberately earning a late booking late in the first leg, and the defense struggled from the start on Tuesday. The attack also faltered again, with the forwards missing chance after chance.
The Bernabeu crowd went silent early when Ziyech opened the scoring after a breakaway that started with a run by Tadic through the right flank.
The already nervous crowd was stunned again about 10 minutes later when Neres added to the lead from close range after Tadic got past a few Real Madrid defenders with a run from near the midfield, clearing Casemiro with a superb spin move and dribble. Neres beat Luka Modric to Tadic's through ball and went past Madrid goalkeeper Thibaut Courtois before finding the open net.
Things got worse for Madrid as both Lucas Vazquez and Vinicius Junior had to be substituted before halftime because of injuries being replaced by Gareth Bale and Marco Asensio.
Vinicius Junior was hurt at the end of a long run in which he nearly opened the scoring for Madrid. The young Brazilian appeared to be crying as he made his way back to the bench.
Bale struck the woodwork with a shot over Ajax goalkeeper Andre Onana just before halftime.
Raphael Varane had hit the crossbar with a header less than five minute into the match, and he also missed from close range later in the first half.
Asensio and Nacho Fernandez also wasted opportunities as Madrid's lack of scoring touch was costly again.
Although the hosts kept pressing, it was Ajax that kept threatening the most throughout the game, missing several chances to increase its lead.
Tadic got his goal with a left-footed shot from inside the area, which was upheld after a lengthy video review to see whether the ball had gone out of play before Ajax launched a quick counterattack.
Asensio gave his team a glimmer of hope in the 70th, but it was snuffed out just two minutes later when Schone's curling free kick from the left flank beat Courtois and sealed the stunning triumph.