PhD Candidate at Boston University
Boston
Joined August 2022
- Tweets11
- Following88
- Followers16
- Likes4
Saad Ullah retweeted
It’s was a crazy Journey through @DARPA’s AIxCC and now using what we’ve learned, we’re trying to ease the flood caused by the vulnpocolypse at Artiphishell! Alongside my amazing cofounders @degrigis @dreselli @_saadullah01 @jay_vadayath /🧵
Cybersecurity teams face thousands of alerts a day but only a few pose real risk. #SCAI doctoral student @cl4sm turned his #AI research into Artiphishell, a $1.5M startup that cuts through alert overload and proves which vulnerabilities matter.
news.engineering.asu.edu/202…
"Can LLMs be used as helpful security assistants for vulnerability detection?"
Our tool "SecLLMHolmes", published at IEEE S&P '24, answers this question by evaluating capabilities of LLMs for identifying and reasoning about security vulnerabilities.
computer.org/csdl/proceeding…
f. Our test cases to assess 'robustness' break chain-of-thought and step-by-step reasoning of LLMs
- gpt-4 fails 17% tests
- palm2 fails 26% tests
- codellama-34b fails 27% tests
e.g., models flip their answers by slight code augmentations
