Matthieu ARZEL

%0 Unpublished work
%T Quantized Guided Pruning for Efficient Hardware Implementations of Convolutional Neural Networks
%+ Département Electronique (ELEC)
%+ Lab-STICC_TB_CACS_IAS
%+ Département Electronique (ELEC)
%+ Département d'Informatique et de Recherche Opérationnelle [Montreal] (DIRO)
%A BOUKLI HACENE, Ghouthi
%A Gripon, Vincent
%A Arzel, Matthieu
%A Farrugia, Nicolas
%A Bengio, Yoshua
%8 2018-12-25
%D 2018
%Z 1812.11337
%K pruning
%K weight binarization
%K Index Terms-convolutional neural networks
%K hardware implementation
%Z Computer Science [cs]/Artificial Intelligence [cs.AI]Preprints, Working Papers, ...
%X Convolutional Neural Networks (CNNs) are state-of-the-art in numerous computer vision tasks such as object classification and detection. However, the large amount of parameters they contain leads to a high computational complexity and strongly limits their usability in budget-constrained devices such as embedded devices. In this paper, we propose a combination of a new pruning technique and a quantization scheme that effectively reduce the complexity and memory usage of convolutional layers of CNNs, and replace the complex convolutional operation by a low-cost multiplexer. We perform experiments on the CIFAR10, CIFAR100 and SVHN and show that the proposed method achieves almost state-of-the-art accuracy, while drastically reducing the computational and memory footprints. We also propose an efficient hardware architecture to accelerate CNN operations. The proposed hardware architecture is a pipeline and accommodates multiple layers working at the same time to speed up the inference process.
%G English
%2 https://hal.archives-ouvertes.fr/hal-01965304/document
%2 https://hal.archives-ouvertes.fr/hal-01965304/file/hardware_quake.pdf
%L hal-01965304
%U https://hal.archives-ouvertes.fr/hal-01965304
%~ CNRS
%~ UNIV-BREST
%~ UNIV-UBS
%~ INSTITUT-TELECOM
%~ ENIB
%~ LAB-STICC_ENIB
%~ LAB-STICC
%~ LAB-STICC_TB
%~ IMT-ATLANTIQUE
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_CACS_IAS
%~ LAB-STICC_IMTA

%0 Journal Article
%T A Fully Flexible Circuit Implementation of Clique-Based Neural Networks in 65-nm CMOS
%+ Institut d’Électronique, de Microélectronique et de Nanotechnologie (IEMN) - UMR 8520 (IEMN)
%+ Département Electronique (ELEC)
%A Larras, Benoît
%A Chollet, Paul
%A LAHUEC, Cyril
%A Seguin, Fabrice
%A Arzel, Matthieu
%< avec comité de lecture
%@ 1549-8328
%J IEEE Transactions on Circuits and Systems I: Regular Papers
%I IEEE
%P 1-12
%8 2018-12-14
%D 2018
%Z Engineering Sciences [physics]/Electronics
%Z Life Sciences [q-bio]/Bioengineering
%Z Life Sciences [q-bio]/Human health and pathology/Cardiology and cardiovascular systemJournal articles
%G English
%L hal-01983523
%U https://hal-imt-atlantique.archives-ouvertes.fr/hal-01983523
%~ IMT-ATLANTIQUE
%~ CNRS
%~ EC-LILLE
%~ INSTITUT-TELECOM
%~ CARDIO
%~ IMTA_ELEC
%~ UNIV-LILLE
%~ UNIV-VALENCIENNES

%0 Journal Article
%T Transfer Incremental Learning Using Data Augmentation
%+ Département Electronique (ELEC)
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Centre de Recherche Administrative de Brest (CRA)
%A Boukli Hacene, Ghouthi
%A Gripon, Vincent
%A Farrugia, Nicolas
%A Arzel, Matthieu
%A Jezequel, Michel
%< avec comité de lecture
%@ 2076-3417
%J Applied Sciences
%I MDPI
%V 8
%N 12
%P 2512
%8 2018-12
%D 2018
%Z Computer Science [cs]/Machine Learning [cs.LG]
%Z Computer Science [cs]/Artificial Intelligence [cs.AI]
%Z Computer Science [cs]/Signal and Image Processing
%Z Computer Science [cs]/Computer Science and Game Theory [cs.GT]Journal articles
%G English
%L hal-01950211
%U https://hal.archives-ouvertes.fr/hal-01950211
%~ CNRS
%~ UNIV-BREST
%~ UNIV-UBS
%~ INSTITUT-TELECOM
%~ ENIB
%~ LAB-STICC
%~ IMT-ATLANTIQUE
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_CACS_IAS
%~ IBSHS
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T A fully flexible circuit implementation of clique-based neural networks in 65-nm CMOS
%+ Institut supérieur de l'électronique et du nunérique (ISEN)
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%A Larras, Benoît
%A Chollet, Paul
%A LAHUEC, Cyril
%A Seguin, Fabrice
%A Arzel, Matthieu
%< avec comité de lecture
%Z 18608
%( Proceedings ISCAS 2018 : IEEE International Symposium on Circuits and Systems (ISCAS)
%B ISCAS 2018 : IEEE International Symposium on Circuits and Systems (ISCAS)
%C Firenze, Italy
%P .
%8 2018-05-27
%D 2018
%R 10.1109/ISCAS.2018.8350954
%K Analogue integrated circuit
%K Neural Networks
%K Energy effciency
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Micro and nanotechnologies/Microelectronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X Clique-based neural networks implement low- complexity functions working with a reduced connectivity be- tween neurons. Thus, they address very specific applications operating with a very low energy budget. This paper proposes a flexible and iterative neural architecture able to implement multiple types of clique-based neural networks of up to 3968 neurons. The circuit has been integrated in a ST 65-nm CMOS ASIC and validated in the context of ECG classification. The network core reacts in 83ns to a stimulation and occupies a 0.21mm 2 silicon area.
%G English
%L hal-01849349
%U https://hal.archives-ouvertes.fr/hal-01849349
%~ INSTITUT-TELECOM
%~ TELECOM-BRETAGNE
%~ CNRS
%~ UNIV-UBS
%~ ENIB
%~ LAB-STICC
%~ UNIV-BREST
%~ IMT-ATLANTIQUE
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_CACS_IAS
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T 40 Gop/S/mm² Fixed-Point Operators for Brain Computer Interface in 65nm CMOS
%+ Lab-STICC_IMTA_MOM_PIM
%+ Département Electronique (ELEC)
%+ Lab-STICC_IMTA_CACS_IAS
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%+ Département Micro-Ondes (MO)
%A Libessart, Erwan
%A Arzel, Matthieu
%A LAHUEC, Cyril
%A Andriulli, Francesco
%< avec comité de lecture
%Z 18759
%B 2018 IEEE International Symposium on Circuits and Systems (ISCAS)
%C Florence, Italy
%P .
%8 2018-05-27
%D 2018
%R 10.1109/ISCAS.2018.8351028
%K BCI
%K Fixed-Point Operators
%K ASIC
%K Reciprocal
%K Inverse square root
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X The performance of non-invasive Brain-Computer Interface (BCI) depends on the computing performance of the system which solves the inverse problem. So the number of basic operations computed per second determines the BCI's resolution. An architecture with pipelined and parallelized flow is then required, and each operator in this architecture must be optimised to reach the highest possible computing performance. This paper presents the implementation of a fixed-point reciprocal and an inverse square root operators for the STMicroelectronics 65 nm CMOS technology. This paper follows previous works that optimise these operators on FPGA target. Each operator reaches a computing performance of about 40 Gop/s/mm², which improves the literature results by a factor of 5. Thus, this works fits well for portable and high performance BCI applications.
%G English
%2 https://hal.archives-ouvertes.fr/hal-01813164/document
%2 https://hal.archives-ouvertes.fr/hal-01813164/file/iscas18.pdf
%L hal-01813164
%U https://hal.archives-ouvertes.fr/hal-01813164
%~ INSTITUT-TELECOM
%~ TELECOM-BRETAGNE
%~ CNRS
%~ UNIV-UBS
%~ ENIB
%~ LAB-STICC
%~ UNIV-BREST
%~ LAB-STICC_IMTA_MOM_PIM
%~ IMTA_ELEC
%~ IMTA_MO
%~ LAB-STICC_IMTA_CACS_IAS
%~ IMT-ATLANTIQUE
%~ LAB-STICC_IMTA

%0 Conference Paper
%F Oral
%T Incremental learning on chip
%+ Département Electronique (ELEC)
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Lab-STICC_TB_CACS_IAS
%A Gripon, Vincent
%A Hacene, Ghouthi Boukli
%A Farrugia, Nicolas
%A Arzel, Matthieu
%A Jezequel, Michel
%< avec comité de lecture
%B 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
%C Montreal, France
%I IEEE
%8 2017-11-14
%D 2017
%R 10.1109/GlobalSIP.2017.8309068
%Z Computer Science [cs]/Machine Learning [cs.LG]
%Z Computer Science [cs]/Artificial Intelligence [cs.AI]
%Z Computer Science [cs]/Signal and Image Processing
%Z Computer Science [cs]/Computer Science and Game Theory [cs.GT]Conference papers
%G English
%L hal-01875912
%U https://hal.archives-ouvertes.fr/hal-01875912
%~ CNRS
%~ UNIV-BREST
%~ UNIV-UBS
%~ INSTITUT-TELECOM
%~ ENIB
%~ LAB-STICC
%~ LAB-STICC_TB
%~ IMT-ATLANTIQUE
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_CACS_IAS
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T Incremental Learning on Chip
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%A Boukli Hacene, Ghouthi
%A Gripon, Vincent
%A Farrugia, Nicolas
%A Arzel, Matthieu
%A Jezequel, Michel
%< avec comité de lecture
%Z 18093
%( Proceedings GlobalSIP 2017 : 5th IEEE Global Conference on Signal and Information Processing - Symposium on Signal Processing for Accelerating Deep Learning
%B GlobalSIP 2017 : 5th IEEE Global Conference on Signal and Information Processing - Symposium on Signal Processing for Accelerating Deep Learning
%C Montréal, Canada
%P .
%8 2017-11-14
%D 2017
%K Field programmable gate arrays
%K Convolutional Neural Network
%K Transfer Learning
%K Incremental Learning
%K Learning on Chip
%Z Computer Science [cs]/Machine Learning [cs.LG]
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X Learning on chip (LOC) is a challenging problem, which allows an embedded system to learn a model and use it to process and classify unknown data, adapting to new obser- vations or classes. Incremental learning of chip (ILOC) is more challenging. ILOC needs intensive computational power to train the model and adapt it when new data are observed, leading to a very difficult hardware implementation. We adress this issue by introducing a method based on the combination of a pre-trained Convolutional Neural Network (CNN) and majority vote, using Product Quantizing (PQ) as a bridge between them. We detail a hardware implementation of the proposed method validated on an FPGA target, with substantial processing acceleration with few hardware resources.
%G English
%2 https://hal.archives-ouvertes.fr/hal-01754847/document
%2 https://hal.archives-ouvertes.fr/hal-01754847/file/GlobalSip_HAL.pdf
%L hal-01754847
%U https://hal.archives-ouvertes.fr/hal-01754847
%~ INSTITUT-TELECOM
%~ TELECOM-BRETAGNE
%~ CNRS
%~ UNIV-UBS
%~ ENIB
%~ LAB-STICC
%~ UNIV-BREST
%~ LAB-STICC_IMTA_CACS_IAS
%~ IMTA_ELEC
%~ IMT-ATLANTIQUE
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T Combining FPGAs and processors for high-throughput forensics
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%+ OVH (Entreprise) (OVH)
%+ Lab-STICC_IMTA_MOM_DIM
%+ Direction de le Recherche et de l'Innovation (DRI)
%A Cornevaux-Juignet, Franck
%A Arzel, Matthieu
%A Horrein, Pierre-Henri
%A Groleat, Tristan
%A Person, Christian
%< avec comité de lecture
%Z 18100
%B CNS 2017 : IEEE Conference on Communications and Network Security
%C Las Vegas, United States
%I IEEE
%P .
%8 2017-10-09
%D 2017
%Z Computer Science [cs]/Cryptography and Security [cs.CR]
%Z Computer Science [cs]/Networking and Internet Architecture [cs.NI]
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X Data centers availability is mandatory and is conditioned by a quick response to failures and attacks thanks to efficient live forensics. However, this task is lately impossible to complete with classic systems because of encountered data rates and service diversity. Moreover, Software-Defined Networking (SDN) devices agility requirements prevent the use of Application Specific Integrated Circuits (ASIC) solutions due to long development time. New solutions of smart Network Interface Cards (NIC) with embedded Field Programmable Gate Arrays (FPGA) are considered, as in Microsoft Azure solution. FPGAs ensure high throughput processings without packet loss to offload CPU processing, but their configurations support only sparse firmware upgrades and shut down processings. This paper proposes an hybrid architecture to realize agile high performance traffic forensics. This work combines hardware performance, high throughput, and software high flexibility to achieve data rates beyond 40 Gb/s while being configurable at runtime through parameters. A software API allows a user-friendly configuration without stopping processings. The implementation of a flexible packet parser, first block of the packet processing chain, demonstrates the viability of the concept.
%G French
%2 https://hal.archives-ouvertes.fr/hal-01742964/document
%2 https://hal.archives-ouvertes.fr/hal-01742964/file/ieee_cns_2017_poster_sub.pdf
%L hal-01742964
%U https://hal.archives-ouvertes.fr/hal-01742964
%~ INSTITUT-TELECOM
%~ TELECOM-BRETAGNE
%~ CNRS
%~ UNIV-UBS
%~ ENIB
%~ LAB-STICC
%~ UNIV-BREST
%~ LAB-STICC_IMTA_CACS_IAS
%~ LAB-STICC_IMTA_MOM_DIM
%~ IMTA_ELEC
%~ IMT-ATLANTIQUE
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T Open-source flexible packet parser for high data rate agile network probe
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%+ OVH (Entreprise) (OVH)
%+ Lab-STICC_IMTA_MOM_DIM
%+ Direction de le Recherche et de l'Innovation (DRI)
%A Cornevaux-Juignet, Franck
%A Arzel, Matthieu
%A Horrein, Pierre-Henri
%A Groleat, Tristan
%A Person, Christian
%< avec comité de lecture
%Z 18101
%( Actes CNS 2017 : IEEE Conference on Communications and Network Security
%B CNS 2017 : IEEE Conference on Communications and Network Security
%C Las Vegas, United States
%P .
%8 2017-10-09
%D 2017
%Z Computer Science [cs]/Cryptography and Security [cs.CR]
%Z Computer Science [cs]/Networking and Internet Architecture [cs.NI]
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X The development of a network centered life has increased overall data rates in core networks. Thus, data centers face the challenge to provide always more services at higher data rates while reacting quickly to complex failures and more powerful attacks thanks to efficient network forensics. Moreover, Software-Defined Networking (SDN) becomes a standard which offers agility but also requires forensic devices able to handle multiple configurations. Although conventional software probes are programmable and thus agile, they cannot support high data rate packet processing any more. Probes could benefit from Application Specific Integrated Circuits (ASIC) to cope with high data rates, but ASICs development time of many months makes them unable to satisfy agility requirements. With reconfiguration ability and high throughput processing without packet loss, Field Programmable Gate Arrays (FPGA) are the key technology chosen by some companies, such as Microsoft, Amazon and OVH, to be integrated into smart Network Interface Cards (NIC). Nevertheless, while high performance criteria is fulfilled, current FPGA probes benefit from an agility still limited to their conventional firmware upgrades which require proprietary tools and hardware-design time and knowledge. This paper proposes the first solution to offer FPGA probes with runtime agility thanks to a flexible packet parser which can be parameterized continuously by a software, endorsing complex tasks and SDN control. This allows a live adaptation of protocol processings from computer host alongside handling packets at line rate without data loss. The proposed parser is open-source and easily usable by network engineers through a Python software API. Benchmark results illustrate the performance of the agile high-level probe implemented on a NetFPGA SUME board, with XC7VX690T FPGA. 60 millions of 64-byte packets are counted based on features provided at runtime. These are selected by the software part, allowing the detection of different volumetric attacks within a few tens of microseconds. This represents a 40 Gb/s traffic of smallest Ethernet packets with no packet loss. With adequate boards, the generic design of the probe offers 160 Gb/s data rates and beyond on modern hardware, assuring probe scalability.
%G French
%2 https://hal.archives-ouvertes.fr/hal-01740903/document
%2 https://hal.archives-ouvertes.fr/hal-01740903/file/ieee_cns_2017_workshop_pub.pdf
%L hal-01740903
%U https://hal.archives-ouvertes.fr/hal-01740903
%~ TELECOM-BRETAGNE
%~ INSTITUT-TELECOM
%~ LAB-STICC_IMTA_CACS_IAS
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_MOM_DIM
%~ LAB-STICC
%~ UNIV-UBS
%~ CNRS
%~ UNIV-BREST
%~ ENIB
%~ IMT-ATLANTIQUE
%~ LAB-STICC_IMTA

%0 Conference Proceedings
%T Budget Restricted Incremental Learning with Pre-Trained Convolutional Neural Networks and Binary Associative Memories
%+ Lab-STICC_IMTA_CACS_IAS
%+ Département Electronique (ELEC)
%+ Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC)
%A Boukli Hacene, Ghouthi
%A Gripon, Vincent
%A Farrugia, Nicolas
%A Arzel, Matthieu
%A Jezequel, Michel
%< avec comité de lecture
%Z 18094
%( Proceedings SIPS 2017 : IEEE International Workshop on Signal Processing Systems
%B SIPS 2017 : IEEE International Workshop on Signal Processing Systems
%C Lorient, France
%P 1 - 4
%8 2017-10-03
%D 2017
%R 10.1109/SiPS.2017.8109978
%K Incremental Learning
%K Transfer Learning
%K Convolutionnal Neural Neworks
%K Associative memory
%Z Computer Science [cs]/Machine Learning [cs.LG]
%Z Computer Science [cs]/Signal and Image Processing
%Z Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV]
%Z Engineering Sciences [physics]/Electronics
%Z Engineering Sciences [physics]/Signal and Image processingConference papers
%X Thanks to their ability to absorb large amounts of data, Convolutional Neural NetThanks to their ability to absorb large amounts of data, Convolutional Neural Networks (CNNs) have become state- of-the-art in numerous vision challenges, sometimes even on par with biological vision. They rely on optimization routines that typically require intensive computational power, thus the question of embedded architectures is a very active field of research. Of particular interest is the problem of incremental learning, where the device adapts to new observations or classes. To tackle this challenging problem, we propose to combine pre-trained CNNs with binary associative memories, using product random sampling as an intermediate between the two methods. The obtained architecture requires significantly less computational power and memory usage than existing counterparts. Moreover, using various challenging vision datasets we show that the proposed architecture is able to perform one-shot learning - and even use only a small portion of the dataset - while keeping very good accuracy.works (CNNs) have become state- of-the-art in numerous vision challenges, sometimes even on par with biological vision. They rely on optimisation routines that typically require intensive computational power, thus the question of embedded architectures is a very active field of research. Of particular interest is the problem of incremental learning, where the device adapts to new observations or classes. To tackle this challenging problem, we propose to combine pre-trained CNNs with binary associative memories, using product random sampling as an intermediate between the two methods. The obtained architecture requires significantly less computational power and memory usage than existing counterparts. Moreover, using various challenging vision datasets we show that the proposed architecture is able to perform one-shot learning - and even use only a small portion of the dataset - while keeping very good accuracy.
%G English
%2 https://hal.archives-ouvertes.fr/hal-01656152/document
%2 https://hal.archives-ouvertes.fr/hal-01656152/file/SIPS.pdf
%L hal-01656152
%U https://hal.archives-ouvertes.fr/hal-01656152
%~ UNIV-BREST
%~ LAB-STICC_IMTA
%~ IMTA_ELEC
%~ LAB-STICC_IMTA_CACS_IAS
%~ LAB-STICC
%~ ENIB
%~ TELECOM-BRETAGNE
%~ UNIV-UBS
%~ CNRS
%~ INSTITUT-TELECOM
%~ IMT-ATLANTIQUE