Apple, a company famed for resisting cooperating with law enforcement and law enforcement wherever possible, regardless of how heinous the crime, is looking to combat child abuse through the use of machine learning software that will actively scan content in iCloud, Siri, and iMessage. At launch, the feature will be opt-in and will not only detect and block any material it determines contain nudity, but it will also alert parents which child has done so.
While this opt-in system seems fairly benign, Apple is also going further by integrating iPhones, iPads, and Macs that will check any images uploaded to the iCloud against a library of known child pornographic materials maintained by the National Center for Missing and Exploited Children (NCMEC). NCMEC and US law enforcement will be alerted of any images detected in this manner, automatically.
Apple has attempted to combat the backlash by stating that any images not flagged will not be visible to Apple, however privacy advocates have argued that Apple should not be searching people’s images in the first place. While this is not a particularly novel concept, as other cloud hosts scan files for dangerous code or illegal content when uploaded, Apple will be the first to continuously scan an individual’s devices and storage.
Apple’s system converts all images to hashes, which are then compared to known hashes maintained by NCMEC, a system that they argue is virtually foolproof, as the hashes are unique to each photo and should be detectable even for an image that has been recolored or cropped. This is where the machine learning comes in, as the hope is that it will be able to determine when a base hash (an original image held by NCMEC) has been altered, thus allowing it to detect images that have been altered. Hypothetically, this may allow an image that has been converted from a digital file to a physical one and then reuploaded to be detected, as the conversion process should produce the same hash. However, this entire process relies on the machine learning algorithm being able to accurately determine what it is looking at. Further, a database of all image hashes is stored both on the user’s device and the cloud and is also encrypted and filled with nonsense data to stop users from interfering with the system.
What makes this proposed process so curious is Apple’s argument that by scanning both the cloud and on-device images they are somehow preserving the privacy of the individual, as opposed to just scanning the cloud images (which they argue would be invasive). This approach raises the question though – if Apple is capable of running an image on a device that is also loaded to iCloud, what’s to stop them from scanning all the images on a device, including the ones which are not uploaded to iCloud?
