In January 1963, Harold Adrian Russell Philby, better known as “Kim,” for the Rudyard Kipling character, escaped to Moscow from Beirut. Philby had lived in the Lebanese capital since August 1956, and prior to that had been one of the most senior figures in MI6, the British Secret Intelligence Service. Yet at the same time he was working as a double agent for the Soviet Union. Suspicions about Philby’s duplicity had surfaced before his departure for Lebanon, forcing him out of MI6, though he had been momentarily exonerated by then-foreign secretary Harold Macmillan. Philby’s friends, feeling he had been wronged, had helped secure for him the post of Middle East correspondent for the Observer and the Economist.

Shortly before the escape in 1963, Nicholas Elliott of MI6 had arrived in Beirut to tell Philby that more recent information had confirmed his guilt. Elliott’s visit to Lebanon was aimed at extracting a confession from Philby. The two were old friends and Elliott had lived in Beirut as MI6 station chief between 1960 and 1962. Before Philby’s departure for Lebanon, Elliott had even managed to put him back on the MI6 payroll, though as a simple agent not an officer.

Many of the details of this story are available in Ben Macintyre’s excellent A Spy Among Friends: Kim Philby and the Great Betrayal. The book has served as a valuable, indeed essential, source of information for this article, as have two remarkable BBC documentaries. The first is based on Macintyre’s book and is titled Kim Philby: His Most Intimate Betrayal, produced for BBC Two in 2014; the second is The Spy Who Went Into the Cold, prepared for BBC Four in 2013. An essay about Philby’s time in Lebanon by our former colleague Tom Carver, written in 2012 for the London Review of Books, filled more gaps. What these sources missed, we tried to compensate for with our own knowledge of Beirut and its recent past.

From our perspective, however, the story of Philby was an excuse to uncover a disappearing facet of our own city, namely that of the 1960s. What we discovered is that Kim Philby’s Beirut is more a thing of the imagination than of reality. Even as we visited some of the key places from his time in the city—his apartment, the old British embassy building and the bar nearby where Philby would drink, the Saint Georges Hotel, the Bashoura cemetery where is father is buried—more often than not nothing remained, or was in ruins.

Yet in many regards the fantasy of Beirut is what is most enduring in a city that, like most cities, is built on illusion. Philby himself had contributed to its legend as a magnet for spies and conspirators of all stripes, sustaining its identity as a romantic place of intrigue. To have seen too much of his city might have dispelled the pleasures of reverie. So Philby’s Beirut took on a life of its own in our minds, because we had to reconstitute it from the broken buildings and transformed neighborhoods that had once served as his home.

When Philby arrived in Lebanon in 1956, he lived for a time not in Beirut but in the mountain village of ‘Ajaltoun, a few kilometers outside Beirut in the Kisirwan district. In those days the village still had a rural quality to it that has changed completely. Philby later moved to an apartment in the Qantari quarter, just behind what was then the presidential palace, which had belonged to the family of Lebanon’s first post-Independence president, Bishara al-Khoury. The quarter is notable for being one of the early areas extending outside Beirut’s walled city, and in Philby’s day was surrounded by beautiful Ottoman-era family palaces that today lie crumbling and are visible from his windows. The building in which Philby and his wife Eleanor lived had once been owned by Qaysar ‘Amer, famous in Lebanon for owning a store that sold toys and fireworks.

Philby’s apartment is not easy to visit. We only managed to enter the building because the owner was generous enough to come down from the mountains and open it up for us. Philby lived on the fifth floor, with a semicircular balcony looking out toward the mountains and the sea. Two photos show the wide angle he once enjoyed, though much of that panorama has now been blocked by buildings.

As we walked through the apartment, we tried to recreate incidents that others had reported upon in Philby’s life. Here is the bathroom, where he stumbled and injured his head only days before fleeing Beirut. There is the balcony ledge from which his pet fox fell, or was pushed off. The animal’s death provoked a surprisingly emotional reaction from Philby, odd from someone who had coldly sent so many people to their death. The kitchen sink, if indeed it is the same one from Philby’s time (and its dated style suggests it may well be), must have been used to wash the countless glasses of whisky that he and Eleanor downed while living in the apartment. And the side balcony is perhaps from where Philby had signaled his Soviet handler when wanting to meet with him.

Philby’s Beirut was much smaller than it is today. Many of his more familiar haunts are within visual range of his semicircular balcony. Let’s begin from left to right. You see that wall of buildings in the foreground? Cut through them and you are heading down to the sea, where you will meet up with Phoenicia Street in the heart of the prewar hotel district. Here you will find the old British embassy building (it’s the one on the left), on the fourth floor of the Spears Building. Until the British moved to a new building on the Corniche in 1960, this was the heart of MI6’s Middle Eastern operations. To this day, the electricity bill arrives in the embassy’s name.

Around the corner from the embassy was a bar where Philby would spend part of his time. What was once Joe’s Bar, located on the ground floor of a now-abandoned building, is only 150 meters from the Phoenicia Hotel, where virtually all the Eurospy films shot in Beirut during the 1960s were set, on what is appropriately called London Street. In an interview for The Spy Who Went Into the Cold, the late historian John Julius Norwich, a young diplomat in Lebanon when Philby lived here, recalled seeing him frequently sitting at a corner table on his own, drinking, “part of the furniture … speechless.”

Philby must have also been speechless a few hundred meters further down from Joe’s Bar, where the renown Saint Georges Hotel is located. The structure is still there, though a dispute between the owner and the company that rebuilt downtown Beirut has prevented the hotel’s reopening. The Saint Georges, where Philby would also stop for drinks, is where he met Eleanor, his third wife. At the time she was married to the New York Times correspondent Sam Brewer and had invited Philby to a table where she was sitting with friends.

Return to Philby’s balcony and look in a diagonal direction, not far beyond the old Saint Elie church visible to the left of the balcony column. At one time that was Beirut’s Jewish quarter of Wadi Abou Jamil, down from which, near the sea, were the Normandy Hotel and Lucullus Restaurant. Both have disappeared and the seaside road on which they were located, once known as the Avenue des Français, was pushed inland by a trash landfill during Lebanon’s civil war. Postwar reconstruction transformed that landfill into gold as some of the most expensive property in Beirut is now located on a portion of it.

The Normandy Hotel was another of Philby’s drinking refuges, while the Lucullus nearby was one of Beirut’s most illustrious restaurants, known for its bouillabaisse and owned by the French Bordes family. When Elliott moved to Beirut in 1960, the first place he took Philby was to Lucullus to celebrate their reunion. Today, only one building from the pre-1975 period remains in that area, on the former seaside stretch visible here, while the neighborhood’s name, Zeitouneh, has been used to brand a nearby marina surrounding the Saint Georges.

Again, go back to Philby’s balcony and imagine what is beyond that ugly, unfinished concrete atrocity in front of you, the Murr tower, blocking your view of the mountains. If you go far enough you will reach an intersection, this side of the Beirut River from the Armenian quarter of Bourj Hammoud. During the 1960s, the Vrej restaurant was located there. It is where Philby would meet Petukhov, his Soviet handler. The building has since been demolished, with a German pub where the restaurant was. Many Lebanese remember Vrej, but they were not likely the kind who could have identified, and perhaps denounced, Philby and his tablemate.

Still on Philby’s balcony? Tack to the right of Murr tower, and in less than a kilometer you will reach Beirut’s historical Bashoura cemetery, final resting place for many of Lebanon’s Sunni families as well as Ottoman governors. It is here that is buried Philby’s father, the redoubtable St. John Philby, a convert to Islam and friend of the first Saudi king, Abdul-Aziz Al Saud. It was St. John’s recommendation that initially led to Kim’s recruitment into the secret service. His death in Beirut in September 1960 was devastating for Kim, sending him off on a round of binge drinking, his customary remedy for sorrow and much else. The cemetery registry has no record of St. John’s grave, which a search also failed to discover, even if it appears certain that he is buried there.

To your right from the balcony, up the road less than ten minutes away is the old British ambassador’s residence on Zarif Street. The beautiful old mansion has since been bought by a charity, but served as the official residence until not very long ago. It is amusing that Philby’s home was roughly at a midpoint between the ambassador’s home and the embassy, on a continuous line at the center of which lived the agent of betrayal.

In the opposite direction of where you are looking, a few kilometers westward of Qantari is the neighborhood of Verdun, where Nicholas Elliott had resided. His apartment was on the top floor of the Tabet Building, long regarded as one of the premier addresses in that part of the city. Today, the building is a bit worn down, though its apartments are being renovated. Sixty years ago, it was probably still possible to see the Mediterranean from there. Macintyre recounts that Elliott, new to the Middle East, relied on Philby to fill him in on the complexities of the region, even as he sent him to Arab countries to gather information for MI6.

Let’s head in the direction of Vrej one again, where nearby, vaguely visible from the Philby home, is the grain silo of the Beirut port. It is from the port that Philby fled Beirut on the rainy night of January 23, 1963, aboard the Dolmatova, a Soviet freighter bound for Odessa. He had called Eleanor, apparently from the Saint Georges, to say that he would be late to a dinner invitation.

A final irony remains in Philby’s building: two pieces of graffiti added much later that somehow seem to encapsulate well what he must have felt as he prepared to leave for the Soviet Union. They can be found on Philby’s landing, one of them an arrow pointing in the direction of an advisory word on the stairway wall.

Next to what had been his apartment door is a less subtle reminder of a thought that surely also crossed his mind, underlined for emphasis.

Finally, after years of thinking about it, Philby made his great escape.