The Chinese president has sworn in the new leader of Hong Kong, Carrie Lam, as the territory marks 20 years since its handover to China from Britain.
But clashes have taken place between pro-democracy and pro-Beijing demonstrators close to the site, with several arrests made.
In a speech, Mr Xi said Hong Kong now enjoyed more freedom than ever before.
It was his first visit to the city since becoming leader in 2013 and comes three years after mass pro-democracy rallies crippled parts of the territory for months.
There in Hong Kong had been “lots of scuffles” between police and pro-democracy protesters on Saturday.
The pro-democracy party, Demosisto, said police had arrested five of its members, and four members from the League of Social Democrats.
Among those said by the group to have been arrested was Joshua Wong, the leader of the so-called umbrella protest movement.
Meanwhile, Mr Xi has overseen the swearing in of the newly-elected chief executive of the territory, Carrie Lam, along with the rest of her cabinet. She is Hong Kong’s first female leader.
“Any attempt to endanger China’s sovereignty and security, challenge the power of the central government… or use Hong Kong to carry out infiltration and sabotage activities against the mainland is an act that crosses the red line and is absolutely impermissible,” he said.
It followed a ceremony in which the flags of China and Hong Kong were raised alongside one another to mark the 20-year anniversary of the city’s handover of British rule.
Helicopters flew overhead as onlookers cheered at the ceremony in Golden Bauhinia Square, central Hong Kong.
On Friday, an official protest zone near the convention centre where Mr Xi was guest of honour at an anniversary banquet and variety performance was heavily patrolled, as demonstrators gathered chanting “end one-party dictatorship”.