Antonio Cinà retweeted
(1/5) Super excited that I will be presenting Conning the Crypto Conman: End-to-End Analysis of Cryptocurrency-based Technical Support Scams on May 20 @IEEESSP - collaboration with fabulous researchers- @5_aad_,@cinofix,@leaschnherr,@hdainguyen,@OestAdam,@pvadrevu,@thorstenholz
Antonio Cinà retweeted
📢 Call for Papers: Workshop on "Human Aligned AI: Towards Algorithms that Humans Can Trust." Discuss trustworthiness in AI, exploring strategies to ensure alignment with human values. Submit by July 31st!
➡️ Conference: icmla-conference.org/icmla24…
➡️ CFP: icmla-conference.org/icmla24…
Antonio Cinà retweeted
Meta’s LLM for software testing work is super exciting.
This paper describes Meta’s TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM’s test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta’s Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement.
Antonio Cinà retweeted
I finally uploaded to YT my 2022 ICML Test of Time Award Talk:
Poisoning Attacks against SVMs: Ten Years After
youtu.be/xWn-bHdAK70?si=PPWh…
Antonio Cinà retweeted
🚨 Fully-funded postdoctoral researcher position at @CISPA to be jointly affiliated with the groups of @BurkholzRebekka and @krikamol.
📰 bit.ly/49N5hTD
🗓️ February 11th, 2024
Both of us will be at @NeurIPSConf. Happy to chat 😀
RT Please 🙏
Antonio Cinà retweeted
𝗗𝗲𝗳𝗶𝗻𝗶𝗻𝗴, 𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗠𝗮𝗻𝗮𝗴𝗶𝗻𝗴 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗗𝗲𝗯𝘁 𝗯𝘆 𝗚𝗼𝗼𝗴𝗹𝗲
In the latest paper by Google Engineers, they researched how to define, measure and manage Technical Debt. They use quarterly engineering satisfaction surveys to analyze the results.
𝟭. 𝗗𝗲𝗳𝗶𝗻𝗶𝘁𝗶𝗼𝗻 𝗼𝗳 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗗𝗲𝗯𝘁
Google took an empirical approach to defining technical debt. They asked engineers about the types of technical debt they encountered and what mitigations would be appropriate to fix this debt. This resulted in a collectively exhaustive and mutually exclusive list of 10 categories of technical debt, including:
🔹 Migration is needed or in progress: This may be motivated by the need for code or systems to be updated, migrated, or maintained.
🔹 Code degradation: The codebase has degraded or not kept up with changing standards over time. The code may be in maintenance mode, needing updates or migrations.
🔹 Documentation on project and application programming interfaces (APIs): Information on how your project works is hard to find, missing, or incomplete.
🔹 Testing: Poor test quality or coverage, such as missing tests or poor test data, results in fragility and flaky tests.
🔹 Code quality: Product architecture or project code must be better designed. It may have been rushed or a prototype/demo.
🔹 Dead and abandoned code: Code/features/projects were replaced or superseded but still need removal.
🔹 Team needs more expertise: This may be due to staffing gaps, turnover, or inherited orphaned code/projects.
🔹 Dependencies: Dependencies are unstable, rapidly changing, or trigger rollbacks.
🔹 Migration could have been better executed or abandoned: This may have resulted in maintaining two versions.
🔹 Release process: The rollout and monitoring of production need to be updated, migrated, or maintained.
𝟮. 𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗗𝗲𝗯𝘁
Google measures technical debt through a quarterly engineering survey. They ask engineers about which of these categories of technical debt have hindered their work. The responses to these surveys help Google identify teams that struggle with managing different types of technical debt. E.g., they found that engineers working on machine learning systems face different types of technical debt compared to engineers who build and maintain back-end services.
They focused on code degradation, teams needing more expertise, and migrations being required or in progress. Then, they explored 117 metrics proposed as indicators of one of these forms of technical debt—the results were that no single metric predicted reports of technical debt from engineers.
𝟯. 𝗠𝗮𝗻𝗮𝗴𝗶𝗻𝗴 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗗𝗲𝗯𝘁
Over the last four years, Google has made a concerted effort to define better, measure, and manage technical debt. Some of the steps taken include:
🔸 Creating a technical debt management framework to help teams establish good practices
🔸 Creating a technical debt management maturity model and accompanying technical debt maturity assessment that
evaluates and characterizes an organization's technical debt management process
🔸 Organizing classroom instruction and self-guided courses to evangelize best practices and community forums to drive continual engagement and sharing of resources.
🔸 Tooling that supports the identification and management of technical debt (for example, indicators of poor test coverage, stale documentation, and deprecated dependencies)
It's important to note that zero technical debt is not the goal at Google. The presence of deliberate, prudent technical debt reflects the practicality of developing systems in the real world. The key is to manage it thoughtfully and responsibly.
Check the full link in the comments.
_______
If you like my posts, please follow me, @milan_milanovic, and hit the 🔔 on my profile to get a notification for all my new posts.
Grow with me 🚀!
#technology #softwareengineering #programming #techworldwithmilan #development
Antonio Cinà retweeted
After almost one year at @CISPA, we decided to call our group the Rational Intelligence (RI) Lab. With machine learning, we aim to create systems that exhibit not only intelligence, but also rationality.
Group website: group.krikamol.org/
RT Appreciated 🙏 (1/3)
Antonio Cinà retweeted
Evaluating LLMs as Agents
-Assesses LLM-as-Agent's reasoning & decision-making abilities
-Multi-turn open-ended generation settings
-8 distinct environments
-Eval of 25 LLMs
-GPT-4 capable across a wide array of real-world tasks
-API-based beats Open
arxiv.org/abs/2308.03688
Antonio Cinà retweeted
📢 Call for papers! Join us at #NeurIPS2023 for the "Backdoors in Deep Learning" workshop 🚀 Submit your cutting-edge research on uncovering vulnerabilities and ensuring AI security. Let's fortify the future of AI together! 🛡️🔐
More details: neurips2023-bugs.github.io/
#AIsecurity #CallForPapers #watermarking #trustworthyAI #backdoor
Antonio Cinà retweeted
Fine-tuning open-source #LLMs like Falcon-7b or GPT-J-6b using @raydistributed is a valuable & reliable option for distributed fine-tuning.
👉 Fine-tune #FALCON-7B w/ #RayAIR, #DeepSpeed & @huggingface lnkd.in/gjPuYAaA
Antonio Cinà retweeted
CLIP has been the backbone of many advances in ML. How vulnerable is CLIP to adversarial attacks?
Presenting CLIPMasterPrints, which are capable of maximizing the confidence score for a number of widely varying prompts, while being unrecognizable for humans🤯
Antonio Cinà retweeted
One month after my cry-out about the absence of good DDP tutorials and I randomly find this hidden gem: github.com/mryab/efficient-d…
Efficient Deep Learning Systems is a whole class on how to debug NNs, optimize their speed, and how **effectively** use distributed. Love it!
Antonio Cinà retweeted
A group of San Franciscans realized that they can disable Waymo and Cruise robotaxis by placing a traffic cone on the vehicle's hood.
They're now encouraging others to do it: "Hell no, we do not consent to this."
Antonio Cinà retweeted
The presentation of the book “The Shortcut" by Nello Cristianini, given at the University of Genova on May 17th 2023 at 15:30, will be also broadcasted on Zoom us02web.zoom.us/j/8247567951… ID riunione: 824 7567 9514 Passcode: 958830
Antonio Cinà retweeted
ChatGPT is used by only 2% of people who have access to the internet.
During this time some people use the right prompts and save a lot of time.
Copy-paste these advanced ChatGPT prompts to get on top of the game:
Antonio Cinà retweeted
With the rise of generative AI tools like ChatGPT, companies are training their algorithms on data that are scraped directly from the open internet. This could leave them vulnerable to “data poisoning” econ.st/40Hnhtz
Antonio Cinà retweeted
LLMs Learning from Natural Language Feedback at Scale
1 LLM generates refinements of LLM output based on feedback
2 LLM selects best refinement
3 Fine-tune LLM to be more likely to chose refinement
-Outperforms fine-tuning on human demonstrations
Paper: arxiv.org/abs/2303.16755
Antonio Cinà retweeted
I don't understand how anyone can believe LLM+plugins won't be a security disaster.
Take a simple app: "GPT4, send emails to people I'm meeting today to say I'm sick"
Sounds useful!
For this, GPT4 needs the ability to read your calendar and send emails.
What could go wrong..?
Antonio Cinà retweeted
Our article "Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference" has been accepted by the journal "Information Sciences".
You can download the final version here: authors.elsevier.com/a/1gjG~… .
#AI @adversarial_ML @trustworthy_ml @adversarial_ML
Antonio Cinà retweeted
📢We are proud to announce that our paper "Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples " has been accepted to #Neurips2022!
w/ @biggiobattista, @zangobot, @ambrademontis, @sotgiu_angelo, Nicholas Carlini, and Fabio Roli
1/n


























