In this blog we will share our experience of using ChatGPT in Aptos CTF MOVEment 2022 competition and discuss the benefits of using this tool in CTF competitions.
CertiK recently took part in the Aptos CTF MOVEment 2022 competition. As with all Capture the Flag (CTF) competitions, we faced the challenge of quickly understanding and summarizing the semantics of complex code snippets. To address this challenge, we turned to ChatGPT, a large language model trained by OpenAI. ChatGPT proved to be a valuable tool in this regard as it could understand and explain the code written in different programming languages.
source
First impressions
Interacting with ChatGPT via natural language allowed us to ask questions about the code and get clear and concise answers. This allowed us to quickly grasp how the code worked, saving valuable time during the CTF competition. For example, when presented with a snippet of code written in Move, ChatGPT can provide a summary of the code’s semantics and explain what the code does in simple, natural language. This can save us a lot of time and effort when trying to quickly understand the code during a CTF contest.
Example 1: Summary code functionality
Here’s an example of how ChatGPT might summarize the functionality of this code snippet provided in the CTF challenge:
public fun pow(g: u128, x: u128, p: u128): u128 {
let ans = 1;
g = g%p;
while (x !=0) {
if ((x & 1) == 1) {
ans = ((ans %p) * (g %p)) %p;
};
x = x >> 1;
g = (g * g) %p;
};
to
}
ChatGPT Summary: The code defines a public function called pow that takes three parameters: g, x, and p. The function calculates and returns the result of raising g to the x power, modulo p. To do this, it uses a loop to repeatedly square g and multiply by ans when the least significant bit of x is 1. The result is then returned. Overall, the code provides a way to calculate the modular exponentiation of g to the xth power, modulo p.
We have been very impressed with the results we have achieved with ChatGPT. Although the code wasn’t particularly long, using ChatGPT made it much quicker to understand and summarize than reading the code directly. This illustrates the potential value ChatGPT can offer to novice programmers. By using ChatGPT, they can quickly get an overview of the code and understand its purpose and structure, which can save them a lot of time and effort.
deep ditch
Although MOVE is a relatively new language, ChatGPT was able to provide us with useful information. We decided to test its capabilities further by trying a slightly more advanced program. Here’s another example where ChatGPT can help explain corner cases that lead to runtime errors and help us debug during code testing.
Example 2: ChatGPT can help to localize a potential runtime error
Q: In what cases does the following code not run?
Fun encrypt_string (plaintext: vector
let key: vector
let key_len : u64 = vector::length(&key);
Leave Ciphertext: Vector
let i = 0;
while ( i < vector::length(&plaintext) ) {
vector::push_back(&mut ciphertext, *vector::borrow(&plaintext, i) ^ *vector::borrow(&key, (i % key_len)));
i = i + 1;
};
ciphertext
}
Response from ChatGPT: The code in the question may fail if transaction_context::get_script_hash() returns an empty vector, or if vector::length() returns 0 when called on plaintext or keys. In either case, the code would not execute the while loop and the function would simply return an empty vector.
While ChatGPT may not be able to identify the exact location of an exception, it can still provide useful insight and descriptions of special cases and potential problems. In this case, ChatGPT might suggest that the transaction_context::get_script_hash() function might return an empty vector, which might cause an error when used in an operation like i % key_len. This information can be useful for someone trying to debug the code and understand possible sources of error.
Handling move bytecode
Besides just summarizing the code semantics, we were also curious if ChatGPT can understand Move bytecode when represented in different formats. We found that ChatGPT Move can understand bytecode when written in text, but is unable to interpret raw bytecode instructions and provide information about their meaning or behavior.
Q: Can you explain the following bytecode?
FunctionDefinition { function: FunctionHandleIndex(1), visibility: private, is_item: false, captures_global_resources: []Code: Some(CodeUnit { locals: SignatureIndex(9), Code: [Call(11), StLoc(3), ImmBorrowLoc(3), CallGeneric(0), ImmBorrowLoc(3), VecLen(11), StLoc(4), VecPack(11, 0), StLoc(1), LdU64(0), StLoc(2), CopyLoc(2), ImmBorrowLoc(0), VecLen(11), Lt, BrFalse(35), Branch(17), MutBorrowLoc(1), ImmBorrowLoc(0), CopyLoc(2), VecImmBorrow(11), ReadRef, ImmBorrowLoc(3), CopyLoc(2), CopyLoc(4), Mod, VecImmBorrow(11), ReadRef, Xor, VecPushBack(11), MoveLoc(2), LdU64(1), Add, StLoc(2), Branch(11), MoveLoc(1), Ret] }) },
Summary of ChatGPT: The code you provided is a function definition written in Move bytecode. The function is defined by the FunctionDefinition block, which contains the function’s function handle, its visibility (in this case Private), whether it is an entry point (in this case false), the global resources it acquires (in this case none), indicates. , and the code of the function.
The function’s code itself is a sequence of bytecode instructions, each of which performs a specific operation. For example, the Call statement calls a function, the StLoc statement stores a value in a local variable, the ImmBorrowLoc statement immutably borrows a local variable, and so on.
It’s difficult to explain the code’s functionality in detail without knowing more about the context in which it is used and the meaning of the various directives and values. In general, however, the code appears to define a function that calls other functions, manipulates local variables, and performs some arithmetic and logical operations. It also uses vector operations like VecLen and VecPushBack and borrows local variables immutable and mutable. Overall, the code seems complex and more information would be needed to understand exactly how it works.
This result is somewhat expected. ChatGPT does not provide an insightful understanding of the code. This is understandable as ChatGPT is a language model and not (yet) specialized enough to understand move bytecode. Despite this, it still generates a meaningful description for the provided program.
limitations
However, it is important to note that ChatGPT has its limitations. It can understand the basics of the Move source code and explain its semantics in general terms, but it is unable to perform detailed logic simulations or provide reliable information about the code’s behavior. In other words, ChatGPT can be a useful tool for quickly understanding code written in Move, but it should not be used for more advanced tasks or for detailed analysis of the code’s behavior.
Example 4: Target branch code analysis failed
Q: When Initialize_balance is set to 10 and initial value of res.balance is set to 10. How can we call the function to leave res.q3 equal to true? (Assuming we can call the function multiple times)
public entry fun add(account: &signer, choice: u8, number: u8) acquires Challenge {
Let res = borrow_global_courage
claim! (number <= 5, 0);
if (choice == 1) {
res.balance = res.balance + number;
} else if (choice == 2) {
res.balance = res.balance * number;
} else if (choice == 3) {
res.balance = res.balance << number;
};
if (!res.q3 && res.balance < Initialize_balance) {
res.q3 = true;
}
}
Response from ChatGPT: To call the add function and set res.q3 to true, you can call the function with choice set to 3 and number set to a value that causes res.balance to be less than Initialize_balance after res.balance by a number has been shifted left bits.
For example, if the initial value of res.balance is 10, you can call the function when choice is 3 and number is 3, which shifts res.balance left by 3 bits, resulting in a value of 80. Since is greater than Initialize_balance, which is set to 10, the if condition in the add function is false and res.q3 remains false.
However, if you call the function with choice set to 3 and number set to 2, res.balance is shifted left by 2 bits, resulting in a value of 40. Since this is less than Initialize_balance, the if condition in the add function becomes true and res.q3 is set to true.
Therefore, to call the function and set res.q3 to true, you can call the add function with choice set to 3 and number set to a value that causes res.balance to be less than Initialize_balance after res.balance moved left by number.
In this case, ChatGPT doesn’t give the right answer and tries to claim that 40 is less than 10.
Conclusion
In summary, our experience of using ChatGPT to resolve CTFs is positive. ChatGPT’s ability to understand and summarize code written in Move was particularly helpful. Although it has its limitations, we believe ChatGPT can be a useful tool for developers working with this language.
By the way, besides the answers provided by ChatGPT, part of this story is written with text generated by ChatGPT. If you include the responses generated by ChatGPT, plug in 90% of this blog content is generated by ChatGPT. While not perfect yet, ChatGPT is an amazing application.
This article is a press release.
What is Certik:
Certik is a blockchain security company that supports projects to identify and eliminate security vulnerabilities in blockchains, smart contracts and Web3 applications with its services, products and cybersecurity techniques.
To learn more about Certik these are the official links:
Website | Twitter | Medium | Telegram | youtube
Learn Crypto Trading, Yield Farms, Income strategies and more at CrytoAnswers
https://nov.link/cryptoanswers
Comments are closed.