A conversational user interface (CUI) is a user interface for computers that emulates a conversation with a human. Historically, computers have relied on text-based user interfaces and graphical user interfaces (GUIs) (such as the user pressing a "back" button) to translate the user's desired action into commands the computer understands. While an effective mechanism of completing computing actions, there is a learning curve for the user associated with GUI. Instead, CUIs provide opportunity for the user to communicate with the computer in their natural language rather than in a syntax specific commands.
Diia
Diia (Ukrainian: Дія [ˈd⁽ʲ⁾ijɐ] , lit. 'Action'; also an acronym for Держава і Я, Derzhava i Ya, IPA: [derˈʒɑwɐ i ˈjɑ], lit. 'State and Me') is a mobile app, a web portal and a brand of e-governance in Ukraine. Launched in 2020, the Diia app allows Ukrainian citizens to use digital documents on their smartphones instead of physical ones for identification and sharing purposes. The Diia portal allows access to over 130 government services. Eventually, the government plans to make all kinds of state-person interactions available through Diia. Diia was built in partnership with the United States and is poised to be shared with other countries. On the sidelines of the 2023 World Economic Forum in Davos, USAID Administrator Samantha Power said the US hopes to replicate the success of Diia in other countries. == History == Diia was first presented on September 27, 2019, by the Ministry of Digital Transformation of Ukraine as a brand of the State in a Smartphone project. Vice Prime Minister and Minister of Digital Transformation Mykhailo Fedorov announced the creation of a mobile app and a web portal that would unite in a single place all the services provided by the state to citizens and businesses. On February 6, 2020, the mobile app Diia was officially launched. During the presentation, Ukrainian President Volodymyr Zelensky said that 9 million Ukrainians now have access to their driver's license and car registration documents on their phones, while Prime Minister Oleksiy Honcharuk called the implementation of the State in a Smartphone project a priority for the government. In April 2020, the Ukrainian government approved a resolution for experimental usage of digital ID-cards and passports which would be issued to all Ukrainians via the Diia. On October 5, 2020, during the Diia Summit, the government presented a first major update of the app and web portal branded "Diia 2.0". More types of documents were added to the app as well as the ability to share documents with others via a single tap on a push-message. The web portal in turn expanded the number of available services to 27, including the ability to register a private limited company in half an hour. President Zelensky who opened the summit, announced that in 2021 Ukraine will enter the "paper less" mode by prohibiting civil servants from demanding paper documents. By the end of 2020, the app had more than six million users, while the portal had 50 available services. In March 2021, the Ukrainian parliament adopted a bill equating digital identity documents with their physical analogues. Starting on August 23, Ukrainian citizens can use digital ID-cards and passports for all purposes while in Ukraine. According to Minister of Digital Transformation Mykhailo Fedorov, Ukraine will become the first country in the world where digital identity documents are considered legally equivalent to ordinary ones. In September 2024, Diia launched an online marriage registration service, which can be beneficial especially for military personnel who spend much time on the frontline separated from their partners. In October 2024, Diia's online marriage service appeared in Time's Inventions of the 2024 list. In the first month of its operations over 1.1 million Ukrainians tried to make proposals using the technology, and 435 couples got married. == Benefits and challenges == The first and most obvious benefit is the convenience of such a platform. Citizens can have many documents on their smartphones at once, without concern about losing or damaging them. Whenever needed, they can just open an app on their smartphones and show/check the document they need. The idea is that Diia will help cut the bureaucracy associated with public services, which in turn will help fight corruption and increase government savings. Fewer people are needed to be employed in the public sector and fewer human to human interactions are supposed to happen. With the start of the program, already 10% of government employees were reduced, which contributes to hundreds of millions of dollars in savings, but besides this, the initiative also improves the speed, efficiency, and transparency of government services. In addition, the digitalization of the government sector helps to develop the whole IT industry in the country, people become more digitally aware and educated, this affects other sectors as well, increasing the spread of digital infrastructure and expediting the speed of overall digitalization. The UN E-government Development Index, which assesses the capabilities of governments to integrate its functions electronically, such as the use of internet and mobile devices, ranked Ukraine 69th in 193 countries surveyed in 2020. Despite its low ranking in the e-government development index, Ukraine made a big jump on the e-participation index, which they ranked 43rd out of 193 countries from 0.66 in 2018 to 0.81 in 2020 (un.org, 2020), suggesting that the government and its citizens are adapting the IT-based government functions. The main goal of e-government according to Perez-Morote et.al. (2020) is to have accountability and transparency among the countries involved. But to do so, there are several challenges that a country should assess first prior to implementing e-government. In the research written by Heeks (2001), the author identified 2 main challenges that countries face in the development of e-government, first is the strategic challenge which involves the preparedness (e-readiness) of the entire government system for electronic transformation, and second challenge is the tactical challenge where the government must design (e-governance design) a system where it can be understood by every user, it's important that the information that needs to be communicated to the consumers is received clearly. For the first challenge (e-readiness), Ukraine had an internet penetration rate of 76% in 2020 and is expected to grow to 82%, it is important that consumers have the internet access for it to enable the consumers to utilize the service. Another factor is the readiness of its institutional infrastructure, which means that the government has its own organization which is solely focused on implementing the e-government project. In the case of Ukraine, the e-governance team is led by Oleksandr Ryzhenko, and the country's e-governance initiative is even further strengthened by ensuring that the data and legal infrastructure are already prepared. Ukraine has done this by modernizing their legislation that is more appropriate in the digital service, and the data exchange solution used by Ukraine is called Trembita. The human infrastructure is also being updated, as competent individuals must be the one doing the task, hence, EGOV4UKRAINE was launched, this aims to get IT developers for developing a system for administrative services. These efforts by the Ukrainian government did not go unnoticed, and they received an award from the e-Governance Academy as "partner of the year 2017". For the second challenge, which deals with the system design, the success of Ukraine can be seen on the latest data of UNDP, where it shows a high increase in the E-participation index. In 2018, Ukraine ranked 75th it ranked 46th in 2020 (un.org, 2020). Despite visible success, the implementation of the e-government was accompanied by problems. Data leakage became the main one. In May 2020, the data of 26 million driver's licenses appeared in the public domain on the Internet. The Ukrainian government said the Diia app was not linked to a data breach, but it is impossible to say for certain. Any storage of official documents in electronic format is associated with the risk of their leakage. In addition, the Diia application still has data protection issues, as the required protection system has not been implemented. This is also compounded by the country's weak data protection legal regime. In addition, since 2023, Ukrainians are able to register their cars with this app. Issued license plates are not using regional codes, but they are using special codes starting with DI or PD. == Diia City == In May 2020, the government presented Diia City headed by Oleksandr Borniakov, a large-scale project which would establish a virtual model of a free economic zone for representatives of the creative economy. It would provide for special digital residency with a particular taxation regime, intellectual property protection and simplified regulations. Diia City concurrently imposes certain constraints on contracts involving individual entrepreneurs (FOPs). It also offers the benefit of tax rebates. Diia City garners endorsement from the Ukrainian government, believing it will support the country's position in the IT market. As of July 30, 2023, the program had more than 600 residents, including companies like iGama, Avenga, SBRobotiks, and Intellectsoft.
Pol.is
Polis (or Pol.is) is wiki survey software designed for large group collaborations. As a civic technology, Polis allows people to share their opinions and ideas, and its algorithm is intended to elevate ideas that can facilitate better decision-making, especially when there are lots of participants. Polis has been credited for assisting the passage of legislation in Taiwan. Pol.is has been used by governments in the United States, Canada, Singapore, Philippines, Finland, Spain and elsewhere. == History == Pol.is was founded by Colin Megill, Christopher Small, and Michael Bjorkegren after the Occupy Wall Street and Arab Spring movements. In Taiwan, pol.is has been "one of the key parts" of vTaiwan's suite of open-source tools for its citizen engagement efforts arising out of the Sunflower Student Movement. vTaiwan claims that of the 26 national issues related to technology discussed on the platform, 80% led to government action. Pol.is is also utilized by "Join," a national platform for online deliberation run by the Taiwanese government. In 2022, Wired reported that Polis was an influence on the Community Notes project at Twitter. In 2023, Megill advised OpenAI on how to facilitate deliberation at scale in a way that was more efficient than Polis, which still required significant human labor and analysis at the time. He helped to award $1 million in grants to teams working on solving the problem of deliberation at scale. In 2023, Anthropic was also exploring steering model behavior using Polis. In 2025, it helped the county that includes Bowling Green, Kentucky make a 25 year plan by facilitating the collection and review of ideas from thousands of residents, representing 10% of the county. 2,370 of 3,940 unique ideas were agreed-upon by over 80% of survey respondents. Ideas were screened by volunteers if they were redundant to an existing idea, off-topic or obscene. == How it works == Pol.is participants are anonymous and cannot reply directly to others posts, in an effort to avoid personal attacks for users. Its algorithms are designed not for engagement and scrolling, but to find areas of agreement to better understand the nuances of a wide range of opinions. Participants are prompted for ideas and vote on other participants' ideas. == Reception == Andrew Leonard, The Financial Times, and VentureBeat describe Pol.is as a possible antidote to the divisiveness of traditional internet discourse by gamifying consensus. Audrey Tang agreed saying, "Polis is quite well known in that it's a kind of social media that instead of polarizing people to drive so called engagement or addiction or attention, it automatically drives bridge making narratives and statements. So only the ideas that speak to both sides or to multiple sides will gain prominence in Polis." Niall Ferguson argues that the approach to utilize tools like Pol.is and Join in Taiwan empowers ordinary people instead of the elite and protects individual freedoms, providing a contrast to the AI-enhanced panopticon model seen in China. Carl Miller praised the technology as having "gamified finding consensus." Darshana Narayanan, in an op-ed in the Economist, argues that open-source machine-learning-based tools like Polis can help to bypass the influence of special interests or experts. Jamie Susskind cited polis and vTaiwan as a model for democracies, particularly around digital policy issues.
Voiceverse NFT plagiarism scandal
In January 2022, 15—the pseudonymous Massachusetts Institute of Technology (MIT) artificial intelligence researcher and creator of the non-commercial generative artificial intelligence voice synthesis research project 15.ai—discovered that the blockchain-based technology company Voiceverse had plagiarized from their platform. Voiceverse marketed itself as a service that offered AI voice cloning technology that could be purchased and traded as non-fungible tokens (NFTs). Amid heightened controversy over NFTs in the gaming industry, voice actor Troy Baker (who has been described as one of the most famous voice actors in video games) announced his partnership with Voiceverse on January 14, 2022, triggering immediate backlash over concerns about the environmental impact of NFTs, potential for fraud, predatory monetization in video games, and the potential of AI displacing jobs for human voice actors. Later that same day, 15 revealed through server logs that Voiceverse had generated voice lines using 15's free text-to-speech platform, pitch-shifted the audio to make them unrecognizable, and falsely marketed the samples as their own technology before selling them as NFTs. Within an hour of being confronted with evidence, Voiceverse confessed and stated that their marketing team had used 15.ai without proper attribution while rushing to create a technology demo to coincide with Baker's partnership announcement, further exacerbating the already negative reception to the original announcement. In response, 15 replied "Go fuck yourself"; the interaction went viral and garnered a large amount of support for the developer. News publications universally characterized this incident as Voiceverse having "stolen" from 15.ai. The next day, Baker appeared on a podcast and stated that his motivation had been to help independent creators who were unable to afford professional voice actors. Following continued backlash and the plagiarism revelation, Baker ended his partnership with Voiceverse on January 31, 2022. Subsequently, the incident was documented in multiple AI ethics databases, criticisms of predatory monetization in video games, and retrospectives as one of the earliest instances of plagiarism and theft stemming from artificial intelligence during the AI boom. == Background == === Troy Baker === Troy Baker is a prominent voice actor in the video game industry best known for his performances as Joel Miller in The Last of Us franchise. Baker has been described as "ubiquitous" by Polygon, "one of the most high-profile and prolific voice actors in video games" by Eurogamer, and "arguably the most famous voice actor in the gaming industry" by GameGuru. His other prominent roles include voicing Agent John "Jonesy" Jones in Fortnite, Booker DeWitt in BioShock Infinite, and both Batman and Joker in multiple Batman video games. As of October 2025, Baker holds the record for the most acting nominations at the BAFTA Games Awards, with five between 2013 and 2021. === Voiceverse === Voiceverse is a blockchain-based startup founded by the Bored Ape Yacht Club that marketed itself as offering AI voice cloning technology in the form of NFTs. Prior to the announcement of their partnership with Baker, Voiceverse had partnered with LOVO, Inc., an AI voice platform that, according to LOVO, could generate human-like voices. Voiceverse stated that any user who purchases a voice NFT would have unlimited and perpetual access to the voice model, which could be used to create content such as audiobooks, YouTube videos, podcasts, e-learning materials, in-game voice chat, and Zoom calls. Voiceverse promised that buyers would "OWN [sic] all of the IP" of content they created using these voices. Voiceverse's roadmap included plans to release 8,888 initial voice NFTs, a feature to add emotions to existing voices, and the ability for users to mint their own voices as NFTs. Prior to Baker's partnership, Voiceverse had also partnered with voice actors Charlet Chung, who voices D.Va in Overwatch, and Andy Milonakis of The Andy Milonakis Show. === 15.ai === 15.ai is a free web application launched in 2020 that uses artificial intelligence to generate text-to-speech voices of fictional characters from popular media. Created by a pseudonymous artificial intelligence researcher known as 15, who began developing the technology as a freshman during their undergraduate research at MIT, it was an early example of an application of generative artificial intelligence during the initial stages of the AI boom. The platform showed that deep neural networks could generate emotionally expressive speech with only 15 seconds of speech; the name "15.ai" references the creator's statement that a voice can be convincingly cloned with just 15 seconds of audio, as opposed to the tens of hours of data previously required. 15.ai became an Internet phenomenon in early 2021 when content utilizing it went viral on social media and quickly gained widespread use among various Internet fandoms. 15 has emphasized that it remain free and non-commercial; it only requires users to give proper credit when using the service for content creation. === NFTs in the video game industry === By early 2022, NFTs had become highly controversial within the gaming industry. Critics raised concerns about their environmental impact due to the significant energy consumption of blockchain technology. In addition, the prevalence of scams, fraud, and potential money laundering associated with NFT sales, as well as fears that NFTs were a new form of predatory monetization following the increasing frequency of loot boxes, caused vocal pushback from the gaming community. Several major gaming companies had begun exploring NFT integration into their products, though fan backlash had already forced some projects to be cancelled. On December 16, 2021, the developers of S.T.A.L.K.E.R. 2: Heart of Chernobyl announced that they would be including NFTs in the game, but cancelled within an hour of the announcement due to immediate universal backlash. Simultaneously, the rise of AI voice technology raised concerns among voice actors about potential job displacement and the devaluation of their work amidst the voice acting industry's ongoing struggles for better compensation and working conditions. == Partnership announcement and backlash == On January 14, 2022, 1:02 a.m. EST, Baker announced on Twitter that he was partnering with Voiceverse "to explore ways where together we might bring new tools to new creators to make new things, and allow everyone a chance to own & invest in the IP's they create." The announcement concluded with the statement "You can hate. Or you can create." Baker's specific role with Voiceverse remained unclear at the time of the announcement. Along with Baker's announcement, Voiceverse promoted their supposed voice AI technology on Twitter by posting animated videos that featured a cat character created by NFT firm Chubbiverse. The videos concluded with text that read "The Voice Powered By Voiceverse"; Voiceverse stated on Twitter that the voices in the animations had been generated using their own AI voice synthesis technology and presented the videos as a technology demonstration of their voice NFT capabilities. The announcement provoked immediate and widespread backlash from the gaming community. Baker's tweet received thousands of replies and quote retweets (the vast majority of which were negative), far more than the number of likes; Michael McWhertor of Polygon described it as a "textbook example of being ratioed" and commented that reactions had been amplified by the final part of Baker's announcement. Michael Beckwith of Metro called Baker's approach "bizarrely aggressive". Later that day, Baker responded to the backlash by apologizing for his choice of words. He said he appreciated people's thoughts and acknowledged that the "hate/create part might have been a bit antagonistic," calling it a "bad attempt to bring levity". Despite the apology, Baker and his fellow voice actors did not distance themselves from Voiceverse at this point. At the same time, Voiceverse attempted to address the criticisms, stating that they were working to move to more environmentally friendly blockchain technology and that voice actors would receive royalties from NFT sales, with actors benefiting from any increase in NFT value. == Plagiarism revelation == On December 13, 2021, amidst the increasingly negative reactions toward NFTs among the general public, the creator of 15.ai (known pseudonymously as 15) announced that they had "no interest in incorporating NFTs into any aspect of [their] work." On January 14, 2022, 11:17 a.m. EST (10 hours after Baker's initial announcement), 15 commented on the Voiceverse venture, stating that it "sounds like a scam". Two hours later, at 1:20 p.m., 15 explicitly accused Voiceverse of "actively attempting to appropriate [15's] work for [Voiceverse's] own benefit." 15 provided evidence through
Open Compute Project
The Open Compute Project (OCP) is an organization that facilitates the sharing of data center product designs and industry best practices among companies. Founded in 2011, OCP has significantly influenced the design and operation of large-scale computing facilities worldwide. As of February 2025, over 400 companies across the world are members of OCP, including Arm, Meta, IBM, Wiwynn, Intel, Nokia, Google, Microsoft, Seagate Technology, Dell, Rackspace, Hewlett Packard Enterprise, NVIDIA, Cisco, Goldman Sachs, Fidelity, Lenovo, Accton Technology Corporation and Alibaba Group. == Structure == The Open Compute Project Foundation is a 501(c)(6) non-profit incorporated in the state of Delaware, United States. OCP has multiple committees, including the board of directors, advisory board and steering committee to govern its operations. As of July 2020, there are seven members who serve on the board of directors which is made up of one individual member and six organizational members. Mark Roenigk (Facebook) is the Foundation's president and chairman. Andy Bechtolsheim is the individual member. In addition to Mark Roenigk who represents Facebook, other organizations on the Open Compute board of directors include Intel (Rebecca Weekly), Microsoft (Kushagra Vaid), Google (Partha Ranganathan), and Rackspace (Jim Hawkins). A list of members can be found on the OCP website. == History == The Open Compute Project began at Facebook (now Meta) in 2009 as an internal project called "Project Freedom". The hardware designs and engineering teams were led by Amir Michael (Manager, Hardware Design) and sponsored by Jonathan Heiliger (VP, Technical Operations) and Frank Frankovsky (Director, Hardware Design and Infrastructure). The three would later open source the designs of Project Freedom and co-found the Open Compute Project. The project was announced at a press event at Facebook's headquarters in Palo Alto on April 7, 2011. == OCP projects == The Open Compute Project Foundation maintains a number of OCP projects, such as: === Server designs === In 2013, two years after the Open Compute Project had started, it was noted that the goal of a more modular server design was "still a long way from live data centers". However, by then some aspects published had been used in Facebook's Prineville data center to improve energy efficiency, as measured by the power usage effectiveness index defined by The Green Grid. Efforts to advance server compute node designs included one for Intel processors and one for AMD processors. Also in 2013, Calxeda contributed a design with ARM architecture processors. Since then, several generations of OCP server designs have been deployed: Wildcat (Intel), Spitfire (AMD), Windmill (Intel E5-2600), Watermark (AMD), Winterfell (Intel E5-2600 v2) and Leopard (Intel E5-2600 v3). === OCP Accelerator Module === OCP Accelerator Module (OAM) is a design specification for hardware architectures that implement artificial intelligence systems that require high module-to-module bandwidth. OAM is used in some of AMD's Instinct accelerator modules. === Rack and power designs === Designs for a mechanical mounting system to replace standard 19-inch racks have been published, with a cabinet the same outside width (600 mm) and depth as existing racks, but with an interior space allowing for wider equipment chassis with a 537 mm width (21 inches). This allows more equipment to fit in the same volume and improves air flow. Compute chassis sizes are defined in multiples of an OpenU or OU, which is 48 mm, slightly taller than the 44 mm rack unit defined for 19-inch racks. As of March 2026, the most current base mechanical definition is the Open Rack V3.1 Specification. At the time the base specification was released, Meta also defined in greater depth the specifications for the rectifiers and power shelf. Specifications for the power monitoring interface (PMI), a communications interface enabling upstream communications between the rectifiers and battery backup unit(BBU) were published by Meta that same year, with Delta Electronics as the main technical contributor to the BBU spec. However, since 2022 the AI boom in the data center has created higher power requirements in order to satisfy the demands of AI accelerators that have been released. As of September 2024, Meta is in the process of updating its Open Rack v3 rectifier, power shelf, battery backup and power management interface specifications to accommodate this increased energy demand. In May 2024, at an Open Compute regional summit, Meta and Rittal outlined their plans for development of their High Power Rack (HPR) ecosystem in conjunction with rack, power and cable partners, increasing power capacity in the rack to 92 kilowatts or more. At the same meeting, Delta Electronics and Advanced Energy reported on their progress in developing new Open Compute standard specifications for power shelf and rectifier designs for HPR applications. Rittal also outlined their collaboration with Meta in designing airflow containment, busbar designs and grounding schemes for the new HPR requirements. === Data storage === Open Vault storage building blocks (also called "Knox") offer high disk densities, with 30 drives in a 2 OU Open Rack chassis designed for easy disk drive replacement. The 3.5 inch disks are stored in two drawers, five across and three deep in each drawer, with connections via serial attached SCSI. There is a "cold storage" variant where idle disks power down to reduce energy consumption. Another design concept was contributed by Hyve Solutions, a division of Synnex, in 2012. At the OCP Summit 2016 Facebook, together with Taiwanese ODM Wistron's spin-off Wiwynn, introduced "Lightning", a flexible NVMe JBOF (just a bunch of flash), based on the existing Open Vault (Knox) design. === Energy efficient data centers === The OCP has published data center designs for energy efficiency. These include power distribution at three-phase 277/480 VAC, which eliminates one transformer stage in typical North American data centers, a single voltage (12.5 VDC) power supply designed to work with 277/480 VAC input, and 48 VDC battery backup. For European (and other 230V countries) datacenters, there is a specification for 230/400 VAC power distribution and its conversion to 12.5 VDC. === Open networking switches === On May 8, 2013, an effort to define an open network switch was announced. The plan was to allow Facebook to load its own operating system software onto its top-of-rack switches. Press reports predicted that more expensive and higher-performance switches would continue to be popular, while less expensive products treated more like a commodity. The first attempt at an open networking switch by Facebook was designed together with Taiwanese ODM Accton using Broadcom Trident II chip and is called "Wedge"; the Linux OS that it runs is called "FBOSS". Later switch contributions include "6-pack" and Wedge-100, based on Broadcom Tomahawk chips. Similar switch hardware designs have been contributed by: Accton Technology Corporation (and its Edgecore Networks subsidiary), Mellanox Technologies, Interface Masters Technologies, Agema Systems. Capable of running Open Network Install Environment (ONIE)-compatible network operating systems such as Cumulus Linux, Switch Light OS by Big Switch Networks, or PICOS by Pica8. A similar project for a custom switch for the Google platform had been rumored, and evolved to use the OpenFlow protocol. === Servers === A sub-project for Mezzanine (NIC) OCP NIC 3.0 specification 1v00 was released in late 2019 establishing three form factors: SFF, TSFF, and LFF. == Litigation == In March, 2015, BladeRoom Group Limited and Bripco (UK) Limited sued Facebook, Emerson Electric Co. and others alleging that Facebook has disclosed BladeRoom and Bripco's trade secrets for prefabricated data centers in the Open Compute Project. Facebook petitioned for the lawsuit to be dismissed, but this was rejected in 2017. A confidential mid-trial settlement was agreed in April 2018.
Attensity
Attensity was an American company that provided social analytics and engagement applications for social customer relationship management (social CRM). Attensity's text analytics software applications extracted facts, relationships and sentiment from unstructured data. == History == Attensity was founded in 2000. An early investor in Attensity was In-Q-Tel, which funds technology to support the missions of the US Government and the broader DOD. InTTENSITY, an independent company that has combined Inxight with Attensity Software (the only joint development project that combines two InQTel funded software packages), was the exclusive distributor and outlet for Attensity in the Federal Market. In 2009, Attensity Corp., then based in Palo Alto, merged with Germany's Empolis and Living-e AG to form Attensity Group. In 2010, Attensity Group acquired Biz360, a provider of social media monitoring and market intelligence solutions. In early 2012, Attensity Group divested itself of the Empolis business unit via a management buyout; that unit currently conducts business under its pre-merger name. Attensity Group was a closely held private company. Its majority shareholder was Aeris Capital, a private Swiss investment office advising a high-net-worth individual and his charitable foundation. Foundation Capital, Granite Ventures, and Scale Venture Partners were among Biz360's investors and thus became shareholders in Attensity Group. In February 2016, Attensity's IP assets were acquired by InContact, and Attensity closed.
Emotion recognition
Emotion recognition is the process of identifying human emotion. People vary widely in their accuracy at recognizing the emotions of others. Use of technology to help people with emotion recognition is a relatively nascent research area. Generally, the technology works best if it uses multiple modalities in context. To date, the most work has been conducted on automating the recognition of facial expressions from video, spoken expressions from audio, written expressions from text, and physiology as measured by wearables. == Human == Humans show a great deal of variability in their abilities to recognize emotion. A key point to keep in mind when learning about automated emotion recognition is that there are several sources of "ground truth", or truth about what the real emotion is. Suppose we are trying to recognize the emotions of Alex. One source is "what would most people say that Alex is feeling?" In this case, the 'truth' may not correspond to what Alex feels, but may correspond to what most people would say it looks like Alex feels. For example, Alex may actually feel sad, but he puts on a big smile and then most people say he looks happy. If an automated method achieves the same results as a group of observers it may be considered accurate, even if it does not actually measure what Alex truly feels. Another source of 'truth' is to ask Alex what he truly feels. This works if Alex has a good sense of his internal state, and wants to tell you what it is, and is capable of putting it accurately into words or a number. However, some people are alexithymic and do not have a good sense of their internal feelings, or they are not able to communicate them accurately with words and numbers. In general, getting to the truth of what emotion is actually present can take some work, can vary depending on the criteria that are selected, and will usually involve maintaining some level of uncertainty. == Automatic == Decades of scientific research have been conducted developing and evaluating methods for automated emotion recognition. There is now an extensive literature proposing and evaluating hundreds of different kinds of methods, leveraging techniques from multiple areas, such as signal processing, machine learning, computer vision, and speech processing. Different methodologies and techniques may be employed to interpret emotion such as Bayesian networks. , Gaussian Mixture models and Hidden Markov Models and deep neural networks. === Approaches === The accuracy of emotion recognition is usually improved when it combines the analysis of human expressions from multimodal forms such as texts, physiology, audio, or video. Different emotion types are detected through the integration of information from facial expressions, body movement and gestures, and speech. The technology is said to contribute in the emergence of the so-called emotional or emotive Internet. The existing approaches in emotion recognition to classify certain emotion types can be generally classified into three main categories: knowledge-based techniques, statistical methods, and hybrid approaches. ==== Knowledge-based techniques ==== Knowledge-based techniques (sometimes referred to as lexicon-based techniques), utilize domain knowledge and the semantic and syntactic characteristics of text and potentially spoken language in order to detect certain emotion types. In this approach, it is common to use knowledge-based resources during the emotion classification process such as WordNet, SenticNet, ConceptNet, and EmotiNet, to name a few. One of the advantages of this approach is the accessibility and economy brought about by the large availability of such knowledge-based resources. A limitation of this technique on the other hand, is its inability to handle concept nuances and complex linguistic rules. Knowledge-based techniques can be mainly classified into two categories: dictionary-based and corpus-based approaches. Dictionary-based approaches find opinion or emotion seed words in a dictionary and search for their synonyms and antonyms to expand the initial list of opinions or emotions. Corpus-based approaches on the other hand, start with a seed list of opinion or emotion words, and expand the database by finding other words with context-specific characteristics in a large corpus. While corpus-based approaches take into account context, their performance still vary in different domains since a word in one domain can have a different orientation in another domain. ==== Statistical methods ==== Statistical methods commonly involve the use of different supervised machine learning algorithms in which a large set of annotated data is fed into the algorithms for the system to learn and predict the appropriate emotion types. Machine learning algorithms generally provide more reasonable classification accuracy compared to other approaches, but one of the challenges in achieving good results in the classification process, is the need to have a sufficiently large training set. Some of the most commonly used machine learning algorithms include Support Vector Machines (SVM), Naive Bayes, and Maximum Entropy. Deep learning, which is under the unsupervised family of machine learning, is also widely employed in emotion recognition. Well-known deep learning algorithms include different architectures of Artificial Neural Network (ANN) such as Convolutional Neural Network (CNN), Long Short-term Memory (LSTM), and Extreme Learning Machine (ELM). The popularity of deep learning approaches in the domain of emotion recognition may be mainly attributed to its success in related applications such as in computer vision, speech recognition, and Natural Language Processing (NLP). ==== Hybrid approaches ==== Hybrid approaches in emotion recognition are essentially a combination of knowledge-based techniques and statistical methods, which exploit complementary characteristics from both techniques. Some of the works that have applied an ensemble of knowledge-driven linguistic elements and statistical methods include sentic computing and iFeel, both of which have adopted the concept-level knowledge-based resource SenticNet. The role of such knowledge-based resources in the implementation of hybrid approaches is highly important in the emotion classification process. Since hybrid techniques gain from the benefits offered by both knowledge-based and statistical approaches, they tend to have better classification performance as opposed to employing knowledge-based or statistical methods independently. A downside of using hybrid techniques however, is the computational complexity during the classification process. === Datasets === Data is an integral part of the existing approaches in emotion recognition and in most cases it is a challenge to obtain annotated data that is necessary to train machine learning algorithms. For the task of classifying different emotion types from multimodal sources in the form of texts, audio, videos or physiological signals, the following datasets are available: HUMAINE: provides natural clips with emotion words and context labels in multiple modalities Belfast database: provides clips with a wide range of emotions from TV programs and interview recordings SEMAINE: provides audiovisual recordings between a person and a virtual agent and contains emotion annotations such as angry, happy, fear, disgust, sadness, contempt, and amusement IEMOCAP: provides recordings of dyadic sessions between actors and contains emotion annotations such as happiness, anger, sadness, frustration, and neutral state eNTERFACE: provides audiovisual recordings of subjects from seven nationalities and contains emotion annotations such as happiness, anger, sadness, surprise, disgust, and fear DEAP: provides electroencephalography (EEG), electrocardiography (ECG), and face video recordings, as well as emotion annotations in terms of valence, arousal, and dominance of people watching film clips DREAMER: provides electroencephalography (EEG) and electrocardiography (ECG) recordings, as well as emotion annotations in terms of valence, dominance of people watching film clips MELD: is a multiparty conversational dataset where each utterance is labeled with emotion and sentiment. MELD provides conversations in video format and hence suitable for multimodal emotion recognition and sentiment analysis. MELD is useful for multimodal sentiment analysis and emotion recognition, dialogue systems and emotion recognition in conversations. MuSe: provides audiovisual recordings of natural interactions between a person and an object. It has discrete and continuous emotion annotations in terms of valence, arousal and trustworthiness as well as speech topics useful for multimodal sentiment analysis and emotion recognition. UIT-VSMEC: is a standard Vietnamese Social Media Emotion Corpus (UIT-VSMEC) with about 6,927 human-annotated sentences with six emotion labels, contributing to emotion recognition research in Vietnamese