In tribute to their achievement, he wrote: "I do not know how they did it, except that they had to—three men of the heroic age of Antarctic exploration with 50feet of rope between them—and a carpenter's adze". After nine months of being beset in ice, they abandoned the badly damaged ship, decamping on to the ice. After failed attempts to march across the ice to this island, Shackleton decided to set up another more permanent camp (Patience Camp) on another floe, and trust to the drift of the ice to take them towards a safe landing. The storms had pushed the James Caird off course, and they had landed on the other side of the island from the whaling station.

They later learned that the same hurricane had sunk a 500-ton steamer bound for South Georgia from Buenos Aires.

In a small boat, the six men spent 16 days crossing 1,300 km of ocean to reach South Georgia and then trekked across the island to a whaling station. The James Caird was launched on 24 April 1916; during the next fifteen days, it sailed through the waters of the southern ocean, at the mercy of the stormy seas, in constant peril of capsizing. After the race to the South Pole ended in December 1911, with Roald Amundsen's conquest, Shackleton turned his attention to the crossing of Antarctica from sea to sea, via the pole.

In the time that passed between abandoning Endurance and watching the ice swallow it up completely, the crew salvaged as many provisions as they could, while sacrificing anything and everything that added weight or would consume valuable resources— including bibles, books, clothing, tools and keepsakes. Mrs Chippy was shot when the Endurance sank, due to the belief that he would not have survived the ordeal that followed. In private, however, he revealed greater foreboding, quietly expressing to the ship’s captain, Frank Worsley, one winter’s night that, “The ship can’t live in this, Skipper … It may be a few months, and it may be only a question of weeks or even days … but what the ice gets, the ice keeps.The plan changed; the destination became the Antarctic, and the project was defined by Shackleton as an "oceanographic and sub-antarctic expedition".

Even though he could no longer be the first to reach the South Pole, Shackleton still wanted to travel there. But Shackleton procured a third ship, the Yelcho, from Chile; and finally, on August 30, 1916, the saga of the Endurance and its crew came to an end. He also socialised with his crew members every evening after dinner, leading sing-alongs, jokes, and games.In August, after several failed attempts, a rescue party set out for Elephant Island, where the remaining 22 crewmen were waiting.