He descended from the famous Yáng family from TaiShan whose antecedents went back to the Eastern Han Dynasty (东汉) where family members served in various senior official posts but did not get elevated into nobility at the time.
The family's star rose during the transition of the Wèi (of Three Kingdoms) to the Western Jìn (西晋) Dynasty (魏晋之际), when they aligned themselves with the Sima (司马) faction. The most famous member of the period was Yáng Hù (羊祜) who was instrumental in laying the groundwork for the Western Jìn to conquer the kingdom of Wu (吴) (of the Three Kingdoms).
After the Turmoils of YongJia Reign (永嘉之乱) which saw the end of the Western Jìn, the Yáng family, along with many other prominent clans, moved south to help establish the Eastern Jìn Dynasty.
Yáng Kăn's grandfather Yáng Guī (羊规) served in the Liú-Sòng Dynasty (劉宋/刘宋) — the Song Dynasty of the Southern Dynasties 南朝, which was ruled by the Liu 劉/刘 Clan. The last years of the dynasty saw fierce internal rivalries which resulted in Yáng Guī following the famous general Xuē ĀnDū (薛安都) to defect to Northern Wèi in AD 468 or AD 469.
[The Liú-Sòng Dynasty of the Southern Dynasties 南朝 was ruled by the Liu 劉/刘 Clan. It was the first of the Southern Dynasties which replaced the Eastern Jìn (东晋), and existed from AD 420-479. Sometimes referred to 南朝宋 Southern Dynasties - Song, also but rare as Southern Song 南宋 as that name is more commonly used to the Southern Song Dynasty which existed in AD 960-1279.]
Yáng Kăn's father Yáng Zhĭ (羊祉, AD 458-516) was a palace secretary and a general in the Northern Wèi. Yáng Zhĭ had a turbulent career in the Northern Wèi which might be the cause of suggestions that he raised his son to desire to return south. (“人生安可久淹异域，汝等可归奉东朝。”)
Yáng Kăn himself rose to be prefect of Taishan under Northern Wèi and defected to Southern Dynasty - Liáng, in AD 528. The Northern Wèi was greatly alarmed and sent one generals after another, including Yü Hui (于晖), Gao Huan (高欢) and Erzhu Yangdu (尔硃阳都) with more than 100,000 troops against Yáng Kăn. Faced with overwhelming odds, and the reinforcements from the Southern Dynasty blocked, Yáng Kăn broke out of the encirclement with his troops. When they reached the southern dynasty's territories, they had more than 10,000 troops and 2,000 horses. Yáng Kăn allowed the troops who were northerners to return rather than force them to relocate. (梁书.列传第三十三 - “卿等怀土，理不能见随，幸适去留，于此别异。”)