It should be clear, first of all, that Pope Paul's teaching opposed Communist ideology. It can be seen as having an origin in the Constitution Gaudium et Spes (nn.20-21), a constitution of the Second Vatican Council that Paul himself promulgated:
Not to be overlooked among the forms of modern atheism is that which anticipates the liberation of man especially through his economic and social emancipation. This form argues that by its nature religion thwarts this liberation by arousing man's hope for a deceptive future life, thereby diverting him from the constructing of the earthly city. Consequently when the proponents of this doctrine gain governmental power they vigorously fight against religion, and promote atheism by using, especially in the education of youth, those means of pressure which public power has at its disposal....
While rejecting atheism, root and branch, the Church sincerely professes that all men, believers and unbelievers alike, ought to work for the rightful betterment of this world in which all alike live; such an ideal cannot be realized, however, apart from sincere and prudent dialogue. Hence the Church protests against the distinction which some state authorities make between believers and unbelievers, with prejudice to the fundamental rights of the human person. The Church calls for the active liberty of believers to build up in this world God's temple too. She courteously invites atheists to examine the Gospel of Christ with an open mind.While Communism is not here condemned or referred to by name, what is intended by this teaching is clear enough in the text - but even more so when the footnotes referring to Pius XI, Pius XII and John XXIII are consulted. Pope Paul VI was to issue his own, more explicit treatment of Communism in 1971, in the Apostolic Letter Octogesima Adveniens n.26:
Therefore the Christian who wishes to live his faith in a political activity which he thinks of as service cannot without contradicting himself adhere to ideological systems which radically or substantially go against his faith and his concept of man. He cannot adhere to the Marxist ideology, to its atheistic materialism, to its dialectic of violence and to the way it absorbs individual freedom in the collectivity, at the same time denying all transcendence to man and his personal and collective history;
And with a balancing teaching against a liberal ideology:
nor can he adhere to the liberal ideology which believes it exalts individual freedom by with drawing it from every limitation, by stimulating it through exclusive seeking of interest and power, and by considering social solidarities as more or less automatic consequences of individual initiatives, not as an aim and a major criterion of the value of the social organization.An evaluation of Vatican Ostpolitik published in 1976 in Keston College's (as it then was) journal Religion in Communist Lands summarised (but do read the whole article as well):
...it has had the following results: first, the persecution of the Church in Poland, Hungary and East Germany has been reduced (this did not apply, however, to the Church in the USSR, Romania and Czechoslovakia); second, the Vatican's anti-communist propaganda has declined; third, a series of exchanges have taken place between papal and communist officials; fourth, Catholics under communist regimes have felt abandoned by the Vatican.From an ecclesiological point of view, one might ask whether the pursuit of direct contacts between the Vatican and Communist regimes which appear, at least at times, to have by-passed the "local churches" in the persons of their bishops was a practise that truly reflected the nature of the "local church" as the presence in a particular place of the universal Church. One might also wonder whether or not the particular genius of Roman Catholicism in having a Supreme Pastor out of the reach of worldly powers able to "confirm in the faith" a local pastor without grace or favour to said worldly powers was adequately expressed.
Ostpolitik certainly had its low points - the humiliation of Cardinal Mindszenty, for example, as he was persuaded to leave Hungary to make possible an agreement between the Vatican and the Hungarian government, and the appointment of a number of inadequate bishops in (then) Czechoslovakia. Another commentator, journalist Desmond O'Grady, identifies a substantial success for Ostpolitik in a chapter of his book The Turned Card. It was the Holy See that proposed that the Helsinki accords on security and cooperation in Europe should acknowledge liberty of conscience and religious freedom as being the right of all peoples. It is worth remembering how influential these accords were for the Human Rights movement in the then- Communist countries in the late 1970's and the 1980's.
The tensions latent in the policy of Ostpolitik were apparent in the particular situation of Cardinal Stefan Wyszynski. In the early pages of his prison notes published under the title A Freedom Wthin, Cardinal Wyszynski explained why he worked towards achieving what was at the time termed "the Mutual Understanding" between the Church in Poland and the Communist authorities, a move that foreshadowed what was later to be undertaken by Cardinal Casaroli as Ostpolitik:
Why did I work towards the Mutual Understanding? From the very beginning, I was and continued to be of the opinion that Poland, like the Church, had lost too much blood during the German occupation to be able to afford to shed any more. It was necessary at any and all costs to stop this process of spiritual bloodshed and return to a normal life so indispensable in the development of the country and the Church.This was the early 1950s. By the time that Cardinal Casaroli visited Poland in 1974 for meetings with the government, there was a clear concern on the part of Cardinal Wyszynski that an agreement might be reached between the Vatican and Poland's government that left unaddressed his concerns about the Church being allowed to play its proper part in Polish society (cf Poland: Troubled Relations between Church and State, an article published in Religion in Communist Lands).
Somewhat as a postscript, one can perhaps observe that a question in the background of the "dialogue" of Ostpolitik is still relevant today. It is the question of the appropriate relation of religious institutions to the instruments of the State, and of religious believers in respect of wider society. Read again Pope Benedict's words in Westminster Hall in 2010, and ask yourself whether or not they might have been applied to the countries of the Communist bloc in the years before 1989:
Religion, in other words, is not a problem for legislators to solve, but a vital contributor to the national conversation. In this light, I cannot but voice my concern at the increasing marginalization of religion, particularly of Christianity, that is taking place in some quarters, even in nations which place a great emphasis on tolerance. There are those who would advocate that the voice of religion be silenced, or at least relegated to the purely private sphere. There are those who argue that the public celebration of festivals such as Christmas should be discouraged, in the questionable belief that it might somehow offend those of other religions or none. And there are those who argue – paradoxically with the intention of eliminating discrimination – that Christians in public roles should be required at times to act against their conscience. These are worrying signs of a failure to appreciate not only the rights of believers to freedom of conscience and freedom of religion, but also the legitimate role of religion in the public square. I would invite all of you, therefore, within your respective spheres of influence, to seek ways of promoting and encouraging dialogue between faith and reason at every level of national life.