Fully Homomorphic Encryption (FHE) enables computations on encrypted data without needing the decryption key, providing significant privacy benefits for neural network applications in sensitive sectors like medicine and finance. However, programming these applications with FHE is challenging and requires deep cryptographic knowledge to ensure correctness, performance, and security.

In this paper, we introduce ACE, the first production-quality open-source FHE compiler developed by a global IT company. ACE automates neural network inference on encrypted data by accepting ONNX models and generating C/C++ programs that use its open-source FHE library. We discuss the design challenges in developing ACE, which aims to support various input formats and architectures across different FHE schemes through an innovative IR supporting multiple abstraction levels. ACE comprises 44K lines of C/C++ code and translates ONNX models into C/C++ for encrypted inference on CPUs, specifically using the RNS-CKKS scheme. Preliminary evaluations on single CPU show that ACE achieves $2.24\times$ speed improvements in ResNet models compared to expert implementations, confirming its effectiveness and meeting our design objectives.