The Chinese ships were sighted about 49 kilometers (30 miles) from the island of Yonaguni, in Japan's Okinawa prefecture (state), according to Japan's Defense Ministry. They were about 200 kilometers (125 miles) from a chain of small islands that have sparked a heated dispute between Japan and China.
The ships were believed to be returning to China after training in the Pacific.
Defense Minister Satoshi Morimoto said Japan is monitoring the ships' movement. Japan considers the area part of its contiguous waters, but it is not illegal for foreign vessels to transit them.
It is not unusual for the Chinese navy to transit waters around Okinawa en route to the Pacific, but this is the first such operation observed this year, according to public broadcaster NHK. The ships included frigates, a guided missile destroyer, a refueler and two submarine rescue vessels.
It was unclear if their mission was directly related to the territorial issue, or whether they were trying to avoid an approaching typhoon.
Japan angered China last month by nationalizing part of a chain of East China Sea islands called Senkaku in Japanese and Diaoyu in Chinese. The move sparked violent protests in China.
Nearby Taiwan also claims the islands, which are uninhabited but surrounded by rich fishing grounds and possibly lucrative undersea energy deposits.
China and Japan have recently stepped up naval activities in the area around Okinawa because of the dispute, but there have been no clashes between their warships, which have generally stayed away from the islands themselves.
Wary of missteps that could lead to a sudden escalation of tensions, the countries have instead sent less threatening coast guard ships. Over the past week, however, both have made a point of showing off their naval prowess.
Chinese websites were abuzz Monday with photographs showing navy pilots practicing touch-and-go landing exercises on China's first aircraft carrier. It wasn't clear when the pictures were taken, and they did not appear on the Defense Ministry's website or in official media.
The carrier was launched last month without aircraft or an accompanying battle group, and actual flight operations could be years away. But it is widely seen as a symbol of China's ambitions to be a leading Asian naval power, especially as it faces sharpening territorial conflicts with Japan and other countries.
Japan's navy, meanwhile, marked its 60th anniversary with a major exercise on Sunday.
About 40 ships - including state-of-the-art destroyers, hovercraft able to launch assaults on rough coastlines and new conventionally powered submarines - took part in Fleet Review 2012, the maritime equivalent of a military parade.
About 30 naval aircraft, mostly helicopters, also participated. For the first time, Japan's navy was joined by warships from the United States, Singapore and Australia. Representatives from more than 20 countries, including China, attended the event staged in waters south of Tokyo