With all wisdom and understanding, he made known to us the mystery of his will according to his good pleasure, which he purposed in Christ, to be put into effect when the times reach their fulfillment— to bring unity to all things in heaven and on earth under Christ (Ephesians 1:7-10).
From this text we learn that the mystery has to do with bringing all things to unity in heaven and earth under Christ. Later we are told that this mystery was not known in the past but came to Paul by means of supernatural revelation:
Surely you have heard about the administration of God’s grace that was given to me for you, that is, the mystery made known to me by revelation, as I have already written briefly. In reading this, then, you will be able to understand my insight into the mystery of Christ, which was not made known to people in other generations as it has now been revealed by the Spirit to God’s holy apostles and prophets (Ephesians 3:2-5).
So when considering this mystery, we cannot speak of it as having been prophesied in the Old Testament to which the New Testament is a fulfillment. It is new.
The key verse is Ephesians 3:6, which tells us straight-way what this mystery is all about:
This mystery is that through the gospel the Gentiles are heirs together with Israel, members together of one body, and sharers together in the promise in Christ Jesus.
The mystery is that Gentiles and Jews would come together as one people of God through the Gospel. It is in this mystery revealed that we understand that the New Covenant, which was given to Jews and was for Jews, also applies to the Gentiles. Paul tells us that this was God's plan all along. God waited until the right time and through His chosen vessel, the church, to reveal it both to those on earth as well as those in heaven:
Although I am less than the least of all the Lord’s people, this grace was given me: to preach to the Gentiles the boundless riches of Christ, and to make plain to everyone the administration of this mystery, which for ages past was kept hidden in God, who created all things. His intent was that now, through the church, the manifold wisdom of God should be made known to the rulers and authorities in the heavenly realms, according to his eternal purpose that he accomplished in Christ Jesus our Lord (Ephesians 3:8-11).
Now, you can understand why there was so mjuch conflict among Christian Jews and Gentiles in the early church and throughout the Book of Acts, so that Paul had to address this issue on numerous occasions and in just about every letter. There was no Old Testament text to validate this truth. Paul received it by revelation (Acts 9). Peter received it by revelation (Acts 10). And the Jews also received it by revelation, namely, the common experience of the baptism of the Holy Spirit confirmed through the sign of tongues (Acts 2, 10).