We talked to prof. dr. sc. Ana Pošćić, President of the EDIH Adria Project Ethics Committee and expert on legal and ethical issues in the context of artificial intelligence. Prof. dr. sc. Pošćić leads a team of interdisciplinary experts dealing with ethical challenges and promoting the highest ethical standards in the development and application of new technologies. In this interview, she will bring us closer to the role of the Ethics Committee, the most current ethical dilemmas in the field of artificial intelligence, and concrete examples of the ethical challenges we face in today's business world.
1. How would you describe the role and responsibilities of the ethics committee whose president you are in the EDIH Adria project?
Through technical expertise, development of innovative services and education, the EDIH ADRIA project ensures the implementation of the highest ethical standards and compliance with legal regulations. In order to support companies in possible doubts about ethical issues, an Ethics Committee was established as an advisory body.
The Board represents a specialised advisory body on ethical and legal issues related to the development and use of AI systems and new technologies. The group consists of experts from various disciplines such as law, computer science, electrical engineering, information sciences, technical sciences, psychology and philosophy. The task of the committee is to identify, consider and reflect on ethical issues related to artificial intelligence, point to sources, principles, disputes, acts of the state and the EU and, depending on needs, possible preparation of guidelines in individual spheres of activity. The concrete result of the activities is the adoption of the Policy on the Use of Artificial Intelligence Tools at the University of Rijeka, in which I actively participated.
2. How does your ethics committee perceive the most current ethical dilemmas that accompany the implementation of artificial intelligence in the business sector today?
There is a lot of talk about AI systems, their advantages, disadvantages and risks, with little or sporadic focus on ethical principles that discuss standards that are right and wrong, acceptable and unacceptable. These are primarily principles, and I think it is important to keep them in mind and think every step forward in the light of the principles and to judge repercussions.
The ethics of AI addresses the important question of how human developers, manufacturers and operators should behave to minimise the ethical harms that AI can cause in society, whether due to poor (unethical) design, inappropriate applications or misuse. Ethical principles in this area address issues such as privacy, system bias, autonomy, liability, but also impact on the labour market and sustainability issues.
As soon as you scratch into this area, you will come across a large number of guides and policies on the responsible and ethical use of AI systems. In particular, a large number have been adopted in the context of the use of generative AI (e.g. World Economic Forum, UNESCO, Council of Europe, EU, etc.).
3. Can you identify concrete examples of ethical challenges or controversies that have recently arisen in the context of the application of AI technology in business?
Russel and Norvig in their textbook ‘Artificial Intelligence: A Modern Approach" summarizes existing ethical principles that can be applied to AI systems such as system security, privacy protection, cooperation of all stakeholders, system transparency, limitation of harmful use, system reliability, accountability, respect for fundamental human rights and values, and respect for diversity/inclusion. In addition to the previous ones, the impact of artificial intelligence on the labour market, the sustainability of the systems themselves and solidarity issues have been increasingly examined. Ethical principles are generally a moral compass and, unlike legal rules, are often more flexible, which should be used as an advantage.
More recently, there are more and more examples where algorithms are used to identify candidates and can inadvertently lead to bias in hiring. An example is discrimination of candidates on the basis of race, gender or some other characteristics. Furthermore, we know that algorithms collect large amounts of data, which increases their efficiency. The collection, storage and analysis of data may raise privacy concerns. Also, machine learning algorithms often operate on the principle of black boxes and sometimes it is not clear (or at least not to the majority) how they make certain decisions. It is precisely the lack of transparency that can make it difficult to establish accountability for results, which ultimately results in a lack of trust on the part of both beneficiaries and the general public.
One of the examples I have recently read about, and it seems to me a bit bizarre, are the so-called ‘deathbots’, which allow users to ‘interact’ with the deceased. Now, with the help of generative tools, anyone who has access to the Internet and with minimal knowledge can "revive" the deceased. Although their primary purpose is to help accept the death of loved ones and adapt to new circumstances, the tools open numerous ethical dilemmas such as the protection of the dignity of the deceased and the issue of data misuse.
4. How do you assess the balance between the potential benefits of AI technologies for companies and the ethical risks they can pose to individuals, society or the environment?
As I have pointed out before, these issues have only been discussed sporadically. Another indicator is a recently published OECD survey on how often terms related to ethics, responsible AI, trustworthy AI are mentioned in job advertisements. In most countries, Less than 1% All posts are mentioned in the previous terms, with an upward trend observed between 2019 and 2022. For example, in Germany, the figure increased from less than 0.05% 2019 at 0.4% in 2022, while for example the UK with 0.1% at 0.4%.
I believe that it is not necessary to list the concrete benefits that new technologies bring to companies in terms of increasing productivity, new innovative products and services, speeding up complex processes and reducing costs. On the other hand, we must not forget about the ethical risks for individuals and society in terms of job loss, reduction of human autonomy, protection of privacy and issues of discrimination. I have to mention another aspect that may be forgotten, which is the development of sustainable systems. Research to date shows that AI systems can contribute to, but also significantly hamper, the achievement of the UN Sustainable Development Goals.
Finally, what is important is the social impact of new technologies, that is, whether it has led to new or deepened existing social inequalities, or to try to direct new technologies to promote social justice and inclusion. All previous risks need to be assessed from an interdisciplinary perspective and appropriate policies, legal frameworks and ethical guidelines need to be developed.
5. What are the key considerations of you and your board regarding issues such as algorithmic bias, violation of privacy or reduction of human autonomy in AI systems used by companies?
Issues such as algorithmic bias, protecting privacy and reducing human autonomy are not a novelty. Problems have existed before. Let us remember only when some ten years ago Amazon developed software to analyze CVs and summarize relevant data in order to rank and search for the best candidates. The system preferred male candidates. I would like to point out that there have been such examples before, but with the emergence of new more advanced tools, the issues are intensifying, which does not mean that systems are now more transparent and that we can explain how systems make decisions.
Another recent example concerns a recent decision of the Court of Justice of the EU (CJEU) in December 2023, where for the first time the Court had the opportunity to clarify and interpret the effect of the provisions of the Regulation on the protection of personal data, more specifically Article 22 of the GDPR. (1). Indeed, the Court ruled that Germany’s largest private credit agency, SCHUFA, unlawfully assesses the creditworthiness of customers on the basis of an automated system based on personal data (profiling). The Court has confirmed that in the previous case, as regards profiling and automated decisions, the mechanisms provided for in the Regulation for the protection of personal data also apply to third parties. This interpretation has an important impact on algorithmic management and the application of automated supervision and decision-making systems in the workplace, although its factual background refers to automated decision-making and evaluation in the context of loan applications.
In the European Union, particular attention has been paid in recent years to a systemic approach in the field of AI. The European Commission has published Ethics Guidelines for Trustworthy Artificial Intelligence, putting people at the heart of the development of artificial intelligence (so-called human-centric artificial intelligence). Trustworthy AI has three components, which need to be fulfilled throughout the entire lifecycle of the system: legality, ethics and resilience. Each component is necessary in itself, but not sufficient to achieve trustworthy AI. Ideally, all three components are coherent and overlap in their actions. The guidelines set out ethical principles and related values that must be respected when developing, deploying and using AI systems, such as respect for human autonomy; prevent the occurrence of damage; justice and explainability. The guidelines detail the requirements that AI systems should meet, specifying the technical and non-technical methods for their implementation. However, it is a horizontal regulation and it will also be necessary to adopt sector-specific regulations in the future, taking into account the specificities of AI systems in different business environments.
6. How do you approach the ethical implications of AI-driven automation, in particular in relation to potential job losses or changes in workforce dynamics?
Effectively reaping the benefits of AI in the labour market requires adequate infrastructure and a skilled workforce. The 2023 Ombudswoman's Report mentions a study by the International Monetary Fund (IMF) according to which artificial intelligence could affect about 60% jobs in advanced economies, while in emerging markets it could change 40%, and in low-income countries 26% jobs. He further states that according to the research of the Croatian Chamber of Economy and Best Advisory d.o.o. in 342 Croatian companies, 50% large companies use some of the artificial intelligence tools, mostly in business process automation, but only 5% companies in the Republic of Croatia regularly educate employees on artificial intelligence.
New processes constantly affect existing employment relationships that lead to new dynamics.
It will be crucial to ensure that workers affected by automation are retrained and retrained to adapt to the new demands of the labour market. First of all, it is important to invest in the education system, change and complement the curricula of individual courses, constantly educate teachers, raise awareness of the advantages and disadvantages, insist on an interdisciplinary approach and provide flexible and adaptable training programs.
Encouraging the development of new jobs and employment opportunities is imperative. In addition, social security issues such as unemployment protection, access to social assistance and healthcare are also important aspects. However, it is necessary to adapt all stakeholders, i.e. to perceive AI tools and systems to assist with existing processes. I believe that AI will not replace humans, but people who use AI tools, but only on the assumption that we are developing controllable systems.
7. What role does your board play in advocating regulatory frameworks or industry standards that promote ethical practices in the use of AI technologies among companies?
As a professor of law, I must address the need for a regulatory framework. While a few decades ago there may have been some doubt as to whether AI systems and new technologies should be regulated at all, today's rapid development imposes the need to consider the approach and method of regulation. On the one hand, we have the accelerated development of new technologies, and on the other hand, the legal system, which is often very slow. In this area, a quick and timely reaction is important.
In order to understand the approach of the European Union, it may be easiest to explain it by example. We know there are automatic facial recognition systems. While, for example, in some countries, such as Poland, serious research is being carried out using these systems to help children with intellectual disabilities communicate with each other and integrate them into society, on the other hand, the same or similar system is being used in China to monitor the population. What I mean is, the system is the same only differs the principle of use. The European Union wants to assert itself as a leader in the field of regulation and in 2021 adopted a proposal for a Regulation on Artificial Intelligence (Artificial Intelligence Act), which was recently adopted. The regulation does not regulate the systems themselves, but their use. The law itself is very detailed and generally points to problems with regulation since, according to some, it is too detailed, and some issues are omitted. It will be seen how the system will evolve in view of rapid changes and difficulties in monitoring regulation. As an example, I note that the first proposal for a Regulation did not include general purpose AI models (generative models) that are now contained in the Regulation.
8. In your opinion, what are the most urgent actions that companies need to take to ensure that their use of AI technologies is in line with ethical principles and values?
Perhaps one of the possible ways is to apply the concept of so-called “ethics by design”, according to which certain ethical principles would already be built into the process of designing systems and technologies. Therefore, ethical aspects should be taken into account at the design stage of the system, and not subsequently or when a problem arises. Already at the product development stage, consideration should be given to the possible implications of products on issues of fairness, transparency, privacy, safety and equity, social and environmental assessment. Furthermore, the above-mentioned Guidelines for Trustworthy AI, addressed to all AI stakeholders who design, develop, deploy, implement, use or are affected by AI, can help. They contain a non-exhaustive list for assessing trustworthy AI that operationalises trustworthy AI. This could be particularly applicable to AI systems that interact directly with users and are primarily intended for developers and deployers of AI systems (whether autonomously developing the system or procuring the system from third parties). It is a tool to support professionals developing AI systems.
Ultimately, AI systems need to be human-centric and based on an obligation to use the system in the service of humanity and the common good.
9. Seven other experts participate in the work of the committee, prof. dr. sc. Sanja Barić, Chair of Constitutional Law, Faculty of Law, University of Rijeka, doc. dr. sc. Ivan Cerovac, Department of Philosophy, Faculty of Humanities and Social Sciences, University of Rijeka, Assoc. prof. dr. sc. Asmir Gračanin, Department of Psychology, Faculty of Humanities and Social Sciences, University of Rijeka, dr. sc. Franko Hržić, University of Rijeka, Faculty of Engineering and Center for Artificial Intelligence and Cybernetic Security, Assoc. prof. dr. sc. Martina Ivašić-Kos, University of Rijeka, Faculty of Informatics and Digital Technologies and Center for Artificial Intelligence and Cybernetic Security, Assoc. prof. dr. sc. Adrijana Martinović, Faculty of Law, University of Rijeka and M.Sc. Damir Medved, Director of the EDIH Adria project, University of Rijeka. How do their competencies and expertise contribute to the work of the board?
Given that these are complex issues that require interdisciplinarity and understanding of various aspects in the work of the committee participate renowned experts in their disciplines who, depending on the need in the light of their expertise, focus on certain aspects.
10. Finally, should we be afraid of the increasing penetration of AI into our business and everyday life?
I believe that we should not be afraid, but approach processes with caution in a way that we will try to understand them, educate ourselves and look at possible positive and negative implications, or try to direct these processes to our benefit, to improve business and business processes. It is important to consider and understand technical, cultural, legal, philosophical and political mechanisms in order to try to control systems and new technologies in a time of rapid change, since this is something that happened centuries and years ago, and now takes place in days, minutes. Mustafa Suleyman also suggests such an approach with a proposal for possible control methods in his most recent book, The Upcoming Wave.
However, even once compliance with legally enforceable fundamental rights is achieved, ethical considerations can help us understand how the development, deployment and use of AI can affect fundamental rights and the values on which they are based, and can help us provide more nuanced guidance when we seek to identify what we should do with technology, rather than what we can (currently) do with it.
It seeks to strike a balance between the promotion of innovation and new technologies and, on the other hand, the need for legal certainty while respecting moral, ethical and sociological standards. I think that the development of new technologies and artificial intelligence systems is desirable, but only as a tool for the benefit of humanity. Investing in education for all of us with an interdisciplinary approach continues to be a priority.
When it comes to Croatia, it seems that we are at the beginning. According to a 2023 OECD survey, an average of 35% In 2021, adults expressed concern that AI will mostly have a negative impact over the next two decades, while 42% He thinks he's going to use it most of the time. In the case of Croatia, less than 30% believes that AI will help or benefit! There is still a lot of work to be done on awareness, and technology and new systems are developing at an exponential rate, or as the title of the book Diamandis and Kotler says: The Future Is Faster Than You Think!
Photo: Archive
