Data inside the enclave is stored safe, unless an unprivileged party gets inside it. Since enclaves get created at the beginning of an app start from the untrusted part of the app, a gateway for attackers could lay here.
Further the SGX on a specific system is only safe, if the secret hardware-based key inside the Memory Encryption Engine is unknown to everybody else as the SGX module itself.
The Processor Reserved Memory gets initiated at the very beginning of starting the system from the BIOS. SGX is only stated safe, if the PRM is inaccessible for other parts of the system besides the CPU. Hardware attacks trough physical ways would be needed for this example.
A malicious enclave can attack software through cache attacks to steal secret information of enclaves. A proof of concept is published in the paper Malware Guard Extension: Using SGX to Conceal Cache Attacks.
Speculative Executions are a plus for the performance but allows to read decrypted enclave data when inside the CPU e.g. in the cache illegally by other software.
Further the given Intel security guidelines must be followed to prevent leakages caused by bad code.