arxiv: v1 [cs.cv] 20 Nov 2018

Transcription

1 Attributing Fake Images to GANs: Analyzing Fingerprints in Generated Images Ning Yu,2 Larry Davis Mario Fritz 3 arxiv:8.0880v [cs.cv] 20 Nov 208 University of Maryland, College Park 2 Max Planck Institute for Informatics, Saarland Informatics Campus 3 CISPA Helmholtz Center i.g., Saarland Informatics Campus Abstract Research in computer graphics has been in pursuit of realistic image generation for a long time. Recent advances in machine learning with deep generative models have shown increasing success of closing the realism gap by using datadriven and learned components. There is an increasing concern that real and fake images will become more and more difficult to tell apart. We take a first step towards this larger research challenge by asking the question if and to what extend a generated fake image can be attribute to a particular Generative Adversarial Networks (GANs) of a certain architecture and trained with particular data and random seed. Our analysis shows single samples from GANs carry highly characteristic fingerprints which make attribution of images to GANs possible. Surprisingly, this is even possible for GANs with same architecture and same training that only differ by the training seed.. Introduction Computer graphics has been for several decades in pursuit of photo realistic image generation. While these approaches have been largely model-based, recent success in machine learning has continued to enrich such approaches with data-driven aspects. In particular, deep learning and generative adversarial networks (GANs) [0] have been contributing to photo realistic image synthesis in hybrid architectures[23] or even fully trained models [3]. However, increasingly concerns manifest that at a certain point fake or manipulated images could become completely indistinguishable from real images [26]. This scenario has severe consequences and can potentially be very disruptive to law, politics and society in general. Therefore, the topic has also received increasing attention in the public [3,, 2]. Therefore, we proposed to investigate whether generated and fake images can be attributed to a source. We hypoth- GAN Typ X Data Y Seed Z GAN Typ A Data B Seed C GAN Typ A Data A Seed A GAN Typ A Data A Seed B Figure. We analyze if fake images can be attributed to a certain GANs model. It turns out that the generated images have inherently characteristic fingerprints even the GANs only differ e.g. by the random seed during training. esize that image generation models carry fingerprints by which the sources will be uniquely identifiable. Equally, such an attribution technique could also be used to detect an unauthorized use of a generator if such generated images would appear e.g. online. In this sense, it can be equally used to protect intellectual property. In order to address the problem of attribution in the context of GANs, we ask several research questions: Can we attribute fake images to GANs? We formalize the problem and present the first approach that associates generated images to the GANs they stem from. What parameters play a role? If images generated by GANs have a characteristic fingerprint, then we are interested in what parameters/inputs such fingerprints depend on. We investigate such parameters as the architecture of the GANs, the training data that was used to train the GANs as well as the random seed that was used to initialize the model training. What type of features does attribution mechanism exploit? Given a classifier architecture for attribution, we are interested in understanding which information the image attribution is possibly derived from. Here we

2 investigate information in different frequency bands, local v.s. global statistics as well as reliance on obvious artifacts v.s. high quality images. In order to achieve the later study, we propose a perceptual ranking method that allows us to automatically generated subsets of images with a high visual quality. How sensitive is the attribution to defense mechanisms? As we are operating in an adversarial setting where our method seeks to attribute fake images, an adversary might want to prevent such attribution. Therefore, we investigate common attack mechanisms that aim at destroying the fingerprints of images. We look at noise, blur, cropping, image compression. While such technique might be effective, we also look at defenses that aim at immunizing our approach to such perturbations. 2. Related Work Generative Adversarial Networks Generative Adversarial Networks (GANs) [0] have dramatically improved sampling form complex, high dimensional, real-world distributions. In particular, they have achieved a remarkable success in image synthesis [0, 2, 6,, 9]. Therefore, we focus on GANs that perform well on synthesizing face appearances, which is not only a well-studied sub-topic but also equally relevant to our initial motivation on distinguishing real v.s. fake images. We do not innovate on the GANs technique - but rather use them as the subject of our study. We choose the following three GANs models as representatives of the current state of the art: ProGAN [3], SNGAN [5], SAGAN [30]. Watermarking of Machine Learning Models. It has been shown that machine learning models can be actively watermarked [3, 28]. In such models, a characteristic pattern can be built into the learned representation, so that at test time it can be differentiated from other potentially similar models. Such watermarking has not been studied for GANs. Also, in our approach we do not assume any active watermarking process. We fully rely on the fingerprints that are inherently to the selected configurations of model architecture, training data and random initialization. Inference Attacks As machine learning models are becoming more ubiquitous, there has been an increasing interest to study attacks on such approaches. In particular, a range of inference attacks has been proposed that allows to infer internal about a machine learning model, while only black box access is assumed. Such technique target parameters [27], hyperparameters [29], architecture [6], information on training data [22, 20] and decision boundaries [8]. There have been first progress to formulate membership inference attacks against generative models too [2]. In contrast, we are not interested in the exact parameters, but rather propose an attribution technique that associates images with the generating GAN. Author Attribution We draw some inspiration from author attribution techniques in the field of natural language processing (NLP) [25, 5]. Authorship attribution and stylometric methods are well established. A range of techniques can be employed to attribute a text to a particular author Earlier techniques rely on hand-crafted and human designed features [4], while more recent techniques are built on deep neural features [7, 9, 7]. We work on a different domain and also our authors are not human - but rather machine learning models. Yet we phrase our identification model in a similar framework and use deep learning techniques as well for attribution. 3. Method Similar to author attribution in NLP, we seek an identification network that is able to identify the origin of a synthesized fake image, which is proposed in subsection 3.. Furthermore, we seek an understanding on which information contributes to discern GANs attribution by proposing a pre-pooling and post-pooling variants of our network in subsection 3.2. The former one assists us in analyzing the reliance on high-frequency information while the latter one is targeting to local patch statistics. Finally, we are also concerned with potential over-fitting of our trained network to highly visible artifacts in a few images that will most likely vanish in future work on GANs networks. Therefore, we introduce a perceptual ranking method in subsection 3.3 that allows us to filter out images with obvious artifacts or defects and thereby selects a challenging subset with high visual quality. 3.. Attribution Network Similar to the authorship attribution task in NLP where an author should be identified upon a given text, we seek a classifier that is able to predict from an image a GANs model that has generated the image. More formally, we are interested in images I sampled from a GANs model that is parameterized by an architecture A, trained from a training set T and initialized using a random seed S: I GAN(A, T, S) () We are interested in an inference mechanism that allows us to reverse the process and predict from an observed image

3 the generating GAN model that is characterized by its 3 parameters: I (A, T, S) (2) In our investigation, we restrict ourselves to a closed world scenario of a set of GANs G = {Y A,T,S = (A, T, S)}. Hence, we cast our problem into a recognition problem over a finite set of labels: I Y A,T,S G (3) We approach this by a Deep Convolutional Neural Network that is trained directly on the image data to predict the choice of a GANs model as a label. Figure 2(a) depicts an overview of our attribution network. Input on the left is a RGB image. The 4-label output with softmax activation is on the right for the cross-entropy classification loss. The intermediate tensor representations are specified by two spatial dimensions followed by the number of channels. 3 3 convolutional filters are used together with a leakyrelu non-linearity. The spatial dimension is gradually reduced by average pooling. This follows a common scheme to () compute an initial feature representation in the second layer, then (2) reduce the overall capacity (by a factor 2 in each layer) until a reasonably sized representation is reached (52 in our case), and (3) end with a fully connected layer to our target labels (4 in our case) Pooling Analysis Networks In order to analyze the features of our attribution network utilizes, we propose a pre-pooling network and a postpooling network. They are modified from the attribution network by replacing the convolutional plus nonlinear activation blocks with pooling operations at strategic locations. The pooling operation systematically reduces the accessible information and thereby allows for conclusions on the learned classification performances. Pre-Pooling Network The level of realism and artifacts for high quality synthesized images could be encoded in high frequency information. Therefore, we seek a tool to analyze the reliance of our attribution network on this type of information. Figure 2(b) shows the architecture of our pre-pooling network. We introduce a Gaussian downsampling layer that eliminates the high frequency content of the image. This enables us to test the hypothesis whether fine details are the main source of information that can act as a fingerprint. In our experiments we control the amount of down sampling in order to tune the effect of the low pass filter. Post-Pooling Network Patch-based statistics have played an important role in modeling natural image statistics (e.g. [24]). Therefore, we also want to test the hypothesis whether attribution can be derived locally based on patch statistics or alternatively whether the overall structure of the image is important. Figure 2(c) shows the architecture of our post-pooling network that first computes an initial feature representation each of them having a certain receptive field that corresponds to a feature for a patch. Afterwards, we perform average pooling across the spatial positions to reduce the spatial information, so that the network has to rely more on the patch statistics. The architecture ends with a fully connected layer. While the figure shows the post-pooling operation for a particular layer, we systematically vary the position of this pooling in our experiments to change from very local patch statistics to more global ones Perceptual Ranking In our evaluation, we are facing the problem that the state-of-the-art GANs approaches are capable of producing highly realistic examples but are equally troubled with severe failure cases with obvious artifacts or defects. We are worried that our classifier performance might be heavily biased due to such simple failure cases. As GANs techniques are improved at a quick pace due to on-going research, our findings might be short-lived and not representative for the overall problem. In order to mitigate such concerns, we investigate a perceptual ranking strategy to filter out failure cases and selects high quality fake images for classifier training and testing. The discriminator in GANs has a similar purpose but unfortunately the obtained scores are not representative for visual quality, as it was already used during training. Otherwise, the generator would have kept improving. Therefore, we introduce an alternative measure that is motivated by a non-parametric distribution of the real data. Given N real data points D real = {x,..., x N }, we can write the the likelihood of a generated sample g given the real data under a non-parametric kernel density estimate as p(g D real ) K(g, x i ), (4) i where K(a, b) is a kernel function e.g. a Gaussian with fixed variance and mean b evaluated at a. Similar to the Naive-Bayes Nearest Neighbor methods [8], we choose to approximate the sum by the nearest neighbor. In practice, this is often a fair assumption - in particular, as most of the elements in the summation will be in the tails of the Gaussian and therefore contribute little. Hence we are left with p(g D real ) K(g, x Nearest Neighbor ) (5) Instead of using the Mahalanobis distance implied by the Gaussian Kernel, we use the pre-trained perceptual distance computed by a deep neural network [32]. We then use the obtained likelihood as a ranking function and select images

4 ing Role of low-frequency features Role of high-frequency feature ture: x3 tion. 28x28x3 28x28x6 64x64x32 32x32x64 6x6x28 8x8x256 4x4x52 xx52 xx4 28x28x3 Gaussian downsample 8x8x3 8x8x256 4x4x52 xx52 xx4 ü Down-sampling to at most 8x8 still leads to ~89% accuracy. ü Pooling afte ~90% accura ü Low frequency info alone contributes to the successful identification. ü High frequen the successfu 28x28x3 28x28x6 64x64x32 avg pool 4x4x32 xx52 xx4 N, (a) Attribution Network Validation Set 40K faces. up to0k a desired from threshold. each of It turns out that this method is very effective {CelebA, in selecting ProGAN, examples SNGAN, of high visual quality among SAGAN} a candidate set. Figure 3 shows on the left a random sample of generated images. On the right, we show random samples of the selected 0% highest ranked images. We observe that the ranking function effectively selects images of higher quality with less artifacts. 4. Experiments 4.. Setup Dataset. We build our analysis upon CelebA dataset [4], a large-scale face attributes dataset with 202, 599 face images that cover large diversities, pose variations, and different backgrounds. CelebA is proposed for several computer vision tasks including face attribute recognition, face detection, and landmark localization, but is also increasingly employed for unsupervised face image generation. We use the crops centered at the the 2st row and 89th column of each image throughout all our experiments. GANs models. In our experiments, image generation is based on three recent state-of-the-art GANs models: Pro- GAN [3] with public code, SNGAN [5] with public code 2, and SAGAN [30] with public code 3. Each model is trained from scratch with their default settings except we unify the number of training epochs to 60, and unify the output size of a generator and the input image size for a discriminator to growing_of_gans 2 projection 3 Self-Attention-GAN-Tensorflow (b) Pre-Pooling Analysis Pre-downscale (c) Post-Pooling Analysis [input 28x28] <G downscale> [res x res] <conv> <fc> [class prob] Post-pool Figure 2. Different Network Architectures. [input 28x28] <conv> [res x res] <pool> <fc Res 28x28 64x64 32x32 6x6 8x8 4x4 Res 28x28 64x64 32x32 6x6 # training: 400k # training: 400k Can we attribute fake images to specific GANs architectures? First, we leverage all the CelebA real-world images to train ProGAN, SNGAN, and SAGAN separately. For the classification task, we configure a balanced training set with 4 classes: {ProGAN, SNGAN, SAGAN, CelebA}. We randomly generate 00, 000 images from each GANs model for the classification training set, and independently generate another 0, 000 images from each GANs model for the testing set. Correspondingly, we randomly pick up 00, 000 images from CelebA for classification training, and pick up another 0, 000 images for testing. In total, we prepare 400, 000 images for training and 40, 000 for testing. Figure Figure 4 shows examples of each data type. In this setup, we achieve an accuracy of 99.50% accuracy. We conclude the fingerprint of a well-train generator architecture can be easily and successfully identified and can be well generalized by a regular CNN to differentiate from other generator architectures as well as from the real dataset used for the training of those generators. We found this result already to be quite remarkable, as no particular effort of watermarking were made. Merely by using different architectures, each images carries an exploitable fingerprint of the GANs architecture What type of information does the attribution network exploit? High vs. low frequency information. We are further curious if band-limited images still contain effective fingerprints for generator architecture classification. We therefore apply our pre-pooling network shown in Figure 2(b) as an analysis tool. Under the same experiment settings, we systematically vary the amount of downsampling by varying the position of the sequence of Gaussian downsampling layers. The following table shows the recognition rates w.r.t. downsampling factors: We conclude, for the current experiment configuration, the low-frequency components of images can singly and still effectively contribute to fingerprint identification un-

5 (a) Random Samples (b) Filtered Samples Figure 3. Despite steady progress in realism of images synthesized by GANs, several state-of-the-art methods still suffer from significant artifacts. We propose a perceptual ranking method that successfully selects high quality examples used as a challenge set in our evaluation. Downsampling Accuracy til the image resolution downsampled to as 28/32 28/32 = 4 4 when the accuracy starts to deteriorate heavily. Surprisingly, images as small as 28/6 28/6 = 8 8 corresponding to very low frequency information still carry effective fingerprints. Patch-based statistics. We additionally push the limit of fingerprint identification towards another direction: instead of a full CNN classifier architecture, we use our postpooling network to intentionally abandon the trainable convolution layers plus the nonlinear activation layers staring from a resolution and the following deeper block(s), which equals to starting early average pooling towards the final fully connected layers at such resolution. As a result, we can evaluate the classification performance on different levels of neural features from a classifier. We vary the pooling resolution according to the following table with the associated results: Resolution Accuracy We conclude, for the current experiment configuration, the low-level neural features of images can singly and still effectively contribute to fingerprint identification as long as there is at least one layer of trainable convolution (corresponding to the early average pooling starting at resolution 64 64). Dependence on artifacts. As described earlier, we are worried that our attribution network focuses on strong artifacts that are still present in several samples drawn from the state-of-the-art GANs. Therefore, we select 0% of the 40, 000 testing samples according to our perceptual ranking function presented in subsection 3.3. As demonstrated in Figure 3, this method is highly effective in selecting high quality examples with little artifacts at least to the human observer. We now evaluate our trained classifier on this subset of testing data and observe an equally strong performance at 99.05% accuracy. This is an indication that our attribution is not based on strong artifacts but works almost equally well on the high quality fake images What roles do training dataset and random initialization play w.r.t. to fingerprinting in the generated images? We further narrow down the investigation to other attributes of GAN models for fingerprint identification, which include the training set for GANs and the randomness for training initialization. For this analysis, we only focus on ProGAN as the representative of generation models. Training Data. We first randomly pick up a subset of CelebA containing 00, 000 images, denoted as CelebA subset. Similarly, we randomly pick up another disjoint subset of CelebA also containing 00, 000 images, denoted as CelebA subset2. We then randomly pick up 50, 000 images from CelebA subset and 50, 000 images from CelebA subset2 to compose a new dataset, denoted as CelebA subset3. It is noticed that CelebA subset3 has 50% overlap with CelebA subset and 50% overlap with CelebA subset2. In the same way, we compose a CelebA subset4 containing 00, 000 images, with 75% overlap with CelebA subset and 25% overlap with CelebA subset2. We collect above datasets aiming at exploring the relationship between the fingerprint identification and the overlaps between dataset subsets.

6 (a) Real CelebA Data (b) ProGAN[3] (c) SNGAN[5] (d) SAGAN[30] Figure 4. Example data used in experiments: (a) real faces from the CelebA dataset; (b) fake faces sampled from a well-trained ProGAN [3]; (c) fake faces sampled from a well-trained SNGAN [5]; and (d) fake faces sampled from a well-trained SAGAN [30]. (a) Plain (b) Noise (c) Blur (d) Crop (e) JPEG Compression Figure 5. In order to test robustness of the proposed attribution scheme, we investigate different image attacks based on the (a) initial plain image: (b) noise, (c) blur, (d) cropping, and (e) JPEG compression. ProGAN subset4 ProGAN subset3 ProGAN subset2 ProGAN subset ProGAN all Data Split CelebA For each subset of CelebA, we train ProGAN and then independently query the well-trained generator with 00, 000 images for classifier training and another 0, 000 images for classifier testing, labeled as ProGAN subset, ProGAN subset2, ProGAN subset3, and ProGAN subset4 separately. The following tables shows the accuracy results for samples of the different model in a multi-class clasification scenario: Accuracy We conclude, for the current experiment configuration, attribution performance remains equally high regardless of strong overlap (up to 75% ) between the training sets of two generator models. The next part will investigate performance for identical training sets (00% overlap) but differentiate models by varying the random seeds. Random initialization. We further narrow down attribution to the random initialization of ProGAN training as an additional attribute for fingerprint identification. We use the same CelebA subset for all models but different random seeds to initialize ProGAN training. The classification accuracy for differentiating 5 random seeds are as follows: Seed Accuracy We conclude for this extreme case that even two generators trained with exactly the same dataset but just different by the training initialization, a classifier is able to extract an accurate fingerprint for attribution. In order to verify our experimental setup, we ran sanity checks. E.g. two identical

7 models trained with the same seed remain indistinguishable and result in random chance performance. We conclude that while simple attacks work, simple adversarial training works equally well to immunize the attribution networks against simple attacks How sensitive is the attribution to attacks? Attacks. We consider four commonly used perturbation methods, as also employed by Oh et al. [7]: noise, blur, crop, and JPEG compression. The intention is to attack the attribution network, so that the fingerprint can no longer be successfully recognized. Examples of the perturbations are shown in Figure 5. Noise adds i.i.d. Gaussian noise of variance 0 2 to testing images; blur performs Gaussian filtering on testing images with kernel size randomly picked from {, 3, 5, 7, 9}; crop crops testing images with a random offset within 0% of the image side lengths and then resizes back to the original, and JPEG compression performs JPEG compression with quality factor 0. Corresponding to the perturbation option of {noise, blur, crop, JPEG compression} on testing images, our pre-trained classifier achieves the following overall accuracies: Attack Noise Blur Crop JPEG Accuracy All the classification performances decrease due to perturbation attacks, but in details: the classifier completely fails over noise and JPEG compression, is still performing in a reasonable manner better than random guess when facing blur attack, and is relatively robust to crop attack. The relatively qualified performance over blur can serve as a side interpretation for the contribution of the low-frequency image components to GANs fingerprint, while the robustness of fingerprint to crop is illustrated as the fact that CelebA dataset covers a sufficient variety of face poses on the image plane. Defenses. In order to counter the attacks and defend our attribution network, we investigate immunizations against the mentioned perturbation attacks. To game with the perturbation attacks as shown above, we further explore the fingerprint identification performances by fine-tuning the classifier assuming the attack is known. It turns out that the updated classifier regains performance and is again effective in attributing the generated images. The accuracy under the immunization option of {noise, blur, crop, JPEG compression} attacks goes back up to roughly the original performance before the attack: Attack & Defense Noise Blur Crop JPEG Accuracy Conclusion We have presented the first method and study of attributing generated images to GANs. It turns out that just a difference in random seeds during training is sufficient to lead to reliable attribution. Also common attacks on the attribution network are not effective, as they can be countered by simple adversarial training with fine-tuning. We provide insight into the exploited information in the attribution network by investigating different frequency bands, local statistics, and an automatically curated subset of high quality images via perceptual ranking. Acknowledgement References This work was supported in part by the DARPA MediFor program under cooperative agreement FA , Physical and Semantic Integrity Measures for Media Forensics. [] Deep fakes: How they are made and how they can be detected. fakes-how-they-are-made-and-how-they-can-be-detected [2] In the age of a.i., is seeing still believing? [3] You thought fake news was bad? deep fakes are where truth goes to die. [4] A. Abbasi and H. Chen. Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace. ACM Transactions on Information Systems (TOIS), 26(2):7, [5] S. Afroz, A. C. Islam, A. Stolerman, R. Greenstadt, and D. McCoy. Doppelgänger finder: Taking stylometry to the underground. In Security and Privacy (SP), 204 IEEE Symposium on, pages IEEE, [6] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein gan. arxiv preprint arxiv: , [7] D. Bagnall. Author identification using multi-headed recurrent neural networks. arxiv preprint arxiv: , [8] O. Boiman, E. Shechtman, and M. Irani. In defense of nearest-neighbor based image classification. In Computer Vision and Pattern Recognition, CVPR IEEE Conference on, pages 8. IEEE,

8 [9] A. Brock, J. Donahue, and K. Simonyan. Large scale gan training for high fidelity natural image synthesis. arxiv preprint arxiv: , [0] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages , 204., 2 [] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems, pages , [2] J. Hayes, L. Melis, G. Danezis, and E. D. Cristofaro. LO- GAN: evaluating privacy leakage of generative models using generative adversarial networks. CoRR, abs/ , [3] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arxiv preprint arxiv:70.096, 207., 2, 4, 6 [4] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), [5] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. arxiv preprint arxiv: , , 4, 6 [6] S. J. Oh, M. Augustin, B. Schiele, and M. Fritz. Towards reverse-engineering black-box neural networks. In ICLR, [7] S. J. Oh, M. Augustin, B. Schiele, and M. Fritz. Towards reverse-engineering black-box neural networks. In Internation Conference on Representation Learning (ICLR), [8] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. Practical black-box attacks against machine learning. In Asia CCS, [9] S. Ruder, P. Ghaffari, and J. G. Breslin. Character-level and multi-channel convolutional neural networks for large-scale authorship attribution. arxiv preprint arxiv: , [20] A. Salem, Y. Zhang, M. Humbert, M. Fritz, and M. Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Annual Network and Distributed System Security Symposium (NDSS), [2] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems (NIPS), pages , [22] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In Security and Privacy (S&P), [23] Z. Shu, E. Yumer, S. Hadap, K. Sunkavalli, E. Shechtman, and D. Samaras. Neural face editing with intrinsic image disentangling. In Computer Vision and Pattern Recognition, 207. CVPR 207. IEEE Conference on, pages. IEEE, 207. [24] E. P. Simoncelli and B. A. Olshausen. Natural image statistics and neural representation. Annual review of neuroscience, 24():93 26, [25] E. Stamatatos. A survey of modern authorship attribution methods. Journal of the Association for Information Science and Technology, 60(3): , [26] S. Suwajanakorn, S. M. Seitz, and I. Kemelmacher- Shlizerman. Synthesizing obama: learning lip sync from audio. ACM Transactions on Graphics (TOG), 36(4):95, 207. [27] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart. Stealing machine learning models via prediction apis. In USENIX Security, [28] Y. Uchida, Y. Nagai, S. Sakazawa, and S. Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 207 ACM on International Conference on Multimedia Retrieval, pages ACM, [29] B. Wang and N. Z. Gong. Stealing hyperparameters in machine learning. In Security and Privacy (S&P), [30] H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena. Selfattention generative adversarial networks. arxiv preprint arxiv: , , 4, 6 [3] J. Zhang, Z. Gu, J. Jang, H. Wu, M. P. Stoecklin, H. Huang, and I. Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 208 on Asia Conference on Computer and Communications Security, pages ACM, [32] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR,