Looking for enclaive's confidential multi-cloud solution. Click here.


How does Enclave Iso­la­tion work? The Case: Intel SGX


In this arti­cle, I recall the con­cept of sep­a­ra­tion of duties in com­put­er sys­tems and its appli­ca­tion to enclaved pro­grams. The con­cept has been imple­ment­ed by Oper­at­ing Sys­tems like Lin­ux and Win­dows to give appli­ca­tions dif­fer­ent priv­i­leges. Rings are a wide­ly deployed OS approach to sep­a­rate priv­i­leges where­by a low­er ring inher­its the priv­i­leges of the high­er ring. Rings play also a car­di­nal role in defin­ing whether a process can access some mem­o­ry area (e.g., the area of a con­cur­rent process).  For exam­ple, Win­dows 7 and Win­dows Serv­er 2008 (and their pre­de­ces­sors) use only two rings, with ring 0 cor­re­spond­ing to ker­nel mode and ring 3 to user mode, because ear­li­er ver­sions of Win­dows ran on proces­sors that sup­port­ed only two pro­tec­tion levels.

With the intro­duc­tion of enclave tech­nolo­gies, being ful­ly mem­o­ry-encrypt­ed process­es, a new type of ring 3 has been intro­duced. It is worth men­tion­ing that enclaves run in user mode.

Mem­o­ry Separation

A process is noth­ing else than a bunch of micro-instruc­tions in cer­tain vir­tu­al address­es / seg­ments. An Intel CPU x86 archi­tec­ture is built to run mul­ti­ple appli­ca­tion soft­ware instances, named process­es. The oper­at­ing sys­tem (OS) allo­cates the com­put­er resources of the run­ning processes. 

In case of cloud com­put­ing, mul­ti­ple oper­at­ing sys­tems could be exe­cut­ed at the same time with the help of an hyper­vi­sor, which man­ages the hard­ware resources between the many oper­at­ing sys­tem instances on the com­put­er. In which sce­nario, the hyper­vi­sor acts like the ker­nel of an Oper­at­ing Sys­tem and man­ages the hard­ware com­put­ing resources between the vir­tu­al­ized oper­at­ing sys­tems (aka vir­tu­al machines). Either of the vir­tu­al­ized oper­at­ing sys­tem may be thought of as “anoth­er process”. As so it allo­cates some mem­o­ry and asks for CPU resources as any oth­er process.

Iso­la­tion is a key fea­ture for every soft­ware. This pro­gram­ming prin­ci­ple has been designed in favour of devel­op­ers to not wor­ry about the inter­ac­tions with oth­er soft­ware. To this end, oper­at­ing sys­tems lever­age the con­cept of vir­tu­al mem­o­ry address­es. That means from the software’s point of view, all the vir­tu­al mem­o­ry on the com­put­er is avail­able for oper­a­tions. In order to imple­ment the vir­tu­al mem­o­ry abstrac­tion, every process will get its own vir­tu­al address space that only ref­er­ences the mem­o­ry allo­cat­ed to that process. The address trans­la­tion con­cept uses a map­ping defined by page tables, which are man­aged by the sys­tem soft­ware, to trans­form a vir­tu­al address to a phys­i­cal address and vice versa.

Address Trans­la­tion Con­cept [1]

As illus­trat­ed below, every process gets its own vir­tu­al address space and its the task of the oper­at­ing sys­tem to mul­ti­plex the sys­tems DRAM between the process­es, while appar­ent­ly the point of view of appli­ca­tion devel­op­ers is, they get access to the whole computer’s DRAM.

Vir­tu­al Mem­o­ry Abstraction

Thus, the iso­la­tion of process­es is achieved and at the same time, it pre­vents appli­ca­tion code to exe­cute mem­o­ry-mapped devices direct­ly. The address trans­la­tion process is car­ried out by a ded­i­cat­ed hard­ware in the CPU, the so called mem­o­ry man­age­ment unit (MMU).

Anoth­er key fea­ture of vir­tu­al­iza­tion is the dis­tinc­tion in soft­ware priv­i­lege lev­els, which are car­ried out by the CPU. A priv­i­lege sep­a­ra­tion imple­ment­ed in hard­ware guar­an­tees, that a soft­ware can­not dam­age oth­er soft­ware indi­rect­ly, by inter­fer­ing with the sys­tem soft­ware man­ag­ing it [1].

The con­cept of priv­i­lege lev­els is hier­ar­chi­cal­ly, means the most priv­i­leged Ring 0 has super­pow­er and the Rings below, are increas­ing­ly less priv­i­leged Rings. That’s the rea­son, why most priv­i­leged lev­els can manip­u­late low­er priv­i­leged lev­els, but not vice versa.

Priv­i­lege lev­els in x86 architecture

For sys­tem design­ers its proven prac­tice to dis­tinct the oper­at­ing sys­tem into a ker­nel (= high priv­i­lege lev­el, Ring 0) and a user-mode (less priv­i­lege lev­el, Ring 3) to achieve a user-safe envi­ron­ment. The ker­nel allo­cates all the hard­ware resources to the oth­er sys­tem com­po­nents (e.g. dri­vers, low­er priv­i­leged process­es) and acts like an API for sys­tem calls (SYSCALLS). The low­est priv­i­lege lev­els are used by stan­dard appli­ca­tions like web brows­er and user appli­ca­tions and is there­for called user-mode in UNIX envi­ron­ments. In win­dows envi­ron­ments, the ker­nel-mode has the syn­onym unpro­tect­ed-mode, because the ker­nel is able to access whole mem­o­ry space. The user-mode is called pro­tect­ed-mode, because the user has low­er pos­si­bil­i­ties to dam­age the sys­tem due to lim­it­ed access to the memory.

Bring­ing Eclaves into this Picture

Enclave exe­cu­tion always hap­pens in pro­tect­ed mode, at ring 3, and uses the address trans­la­tion set up by the OS ker­nel and hyper­vi­sor. To avoid leak­ing pri­vate data, a CPU that is exe­cut­ing enclave code does not direct­ly ser­vice an inter­rupt, fault (e.g., a page fault) or VM exit. Instead, the CPU first per­forms an Asyn­chro­nous Enclave Exit to switch from enclave code to ring 3 code, and then ser­vices the inter­rupt, fault, or VM exit. 

The CPU per­forms an AEX by sav­ing the CPU state into a pre­de­fined area inside the enclave and trans­fers con­trol to a pre-spec­i­fied instruc­tion out­side the enclave, replac­ing CPU reg­is­ters with syn­thet­ic val­ues. The allo­ca­tion of enclave page cache (EPC) pages to enclaves is del­e­gat­ed to the OS ker­nel (or hyper­vi­sor). The OS com­mu­ni­cates its allo­ca­tion deci­sions to the SGX imple­men­ta­tion via spe­cial ring 0 CPU instruc­tions. The OS can also evict EPC pages into untrust­ed DRAM and lat­er load them back, using ded­i­cat­ed CPU instruc­tions. SGX uses cryp­to­graph­ic pro­tec­tions to assure the con­fi­den­tial­i­ty, integri­ty and fresh­ness of the evict­ed EPC pages while they are stored in untrust­ed memory.

Contact us

Cookie Consent with Real Cookie Banner