-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core: Store object name separately for symbols #13113
Conversation
@dolphin-emu-bot rebuild |
I'm changing this to draft as I think it would be better to also include code to handle the file name instead of just discarding it. I would like to also update the symbol view accordingly to still show file names but that might be better as a separate PR so I'm open to discussions on that. |
Ok, ready for review again. |
I suspect that this change can break other symbol map formats. Moreover, the object might not necessarily be a IIRC, the stripped symbol name is already available through the |
Those are fair concerns; I do recall some map files having .c, .s, etc at the end. However, it would be very easy to remedy that issue - all that would need to be done is just to pick the last part of the split string, as it will always be the object file. I would be willing to test it against various map formats if so desired; I think that it would be a shame to just settle for a GUI toggle instead of being able to develop on top of this, perhaps adding the option to filter by object file (something I also want to try in the future if this is accepted). I feel that the debugging features in Dolphin have a lot of room for improvement and while I understand that this definitely needs to be made more robust, I would be willing to put in the necessary work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your changes are going way beyond just adding the object name properties. You're changing the way symbol maps are loaded and saved.
I couldn't test it because it doesn't compile but I highly suspect that regular expressions might have a performance impact compared to the original code. The way you're matching for the object name might also be too strict.
IMO, I consider changing the name
value not fine, as it forces you to change the logic at so many places. For instance, all those ternary could have been avoided.
In sum, I'd advise you to add an object_name
member as a first PR. Then, build features on top of it with follow-up PRs.
Given how large the PR has grown, I suppose it is reasonable that only the object name split changes are included in this PR, and the rest can be considered after. That being said, I did timing tests, and this is what I got:
So, needless to say, there is definitely lots of room for improvement. |
I decided to essentially revert the regex changes for this PR, and add a few additional checks to handle getting the actual symbol name and object name. The performance is only slightly worse than the current mainline branch:
|
e701825
to
428032b
Compare
3e853bb
to
568779d
Compare
@dolphin-emu-bot rebuild |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested it with some symbol maps and it doesn't play well with them.
Could you please squash your commits into a single one and address the buildbot lint and build failures?
std::string line = fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name); | ||
// Also write the object name if it exists | ||
if (!symbol->object_name.empty()) | ||
line += fmt::format(" {0}", symbol->object_name); | ||
line += "\n"; | ||
f.WriteString(line); | ||
} | ||
|
||
// Write .data section | ||
f.WriteString("\n.data section layout\n"); | ||
for (const auto& symbol : data_symbols) | ||
{ | ||
// Write symbol address, size, virtual address, alignment, name | ||
f.WriteString(fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}\n", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name)); | ||
std::string line = fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name); | ||
// Also write the object name if it exists | ||
if (!symbol->object_name.empty()) | ||
line += fmt::format(" {0}", symbol->object_name); | ||
line += "\n"; | ||
f.WriteString(line); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some symbols have spaces when not mangled, so separating the symbol name and the object name with a space instead of a tab (like codewarrior symbol map) will most likely break Dolphin's parsing when loading these saved map.
Zelda TP:
00000000 000054 80006920 4 SECallback(int, int) d_home_button.o
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part isn't addressed. I highly recommend you to separate the symbol name and its object with a tab. Some symbols have multiple objects (e.g. AIInit<space><tab>ai.a<space>ai.o
).
9a94563
to
ce68a03
Compare
@dolphin-emu-bot rebuild |
std::string line = fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name); | ||
// Also write the object name if it exists | ||
if (!symbol->object_name.empty()) | ||
line += fmt::format(" {0}", symbol->object_name); | ||
line += "\n"; | ||
f.WriteString(line); | ||
} | ||
|
||
// Write .data section | ||
f.WriteString("\n.data section layout\n"); | ||
for (const auto& symbol : data_symbols) | ||
{ | ||
// Write symbol address, size, virtual address, alignment, name | ||
f.WriteString(fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}\n", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name)); | ||
std::string line = fmt::format("{0:08x} {1:08x} {2:08x} {3} {4}", symbol->address, symbol->size, | ||
symbol->address, 0, symbol->name); | ||
// Also write the object name if it exists | ||
if (!symbol->object_name.empty()) | ||
line += fmt::format(" {0}", symbol->object_name); | ||
line += "\n"; | ||
f.WriteString(line); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part isn't addressed. I highly recommend you to separate the symbol name and its object with a tab. Some symbols have multiple objects (e.g. AIInit<space><tab>ai.a<space>ai.o
).
b911754
to
225481a
Compare
@dolphin-emu-bot rebuild |
std::vector<std::string> parts = SplitString(name, '\t'); | ||
size_t num_parts = parts.size(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't they be marked as const
?
5ac2ac7
to
3ab07c9
Compare
3ab07c9
to
97931a7
Compare
9e1910c
to
63ffc4a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Were those leading spaces change intentional? If so, please make such change appear in the commit message and/or in the GitHub conversation as they can be missed quite easily. If they weren't intentional, please make sure you double-check your change before pushing them.
Otherwise, you should also squash your commits into a single one.
The leading spaces are intentional. |
c2bf177
to
55ac242
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This leading spaces change will likely break tools designed to load Dolphin generated symbol maps, especially if they don't take into account leading spaces.
There exist symbol maps generated without these leading spaces. However, I can't confirm that these were generated with CW.
I understand there likely are some tools that might break from this, however there also are many tools that account for, or even assume the CodeWarrior symbol map format, which always includes leading spaces before symbols. For example, https://github.com/Cuyler36/Ghidra-GameCube-Loader/ does not support Dolphin symbol maps as is. |
IIRC, Dolphin already provides scripts to save/load symbol maps from/to IDA/Ghidra in Tools. Your PR is changing how symbol maps are saved, which wasn't the initial goal of your PR, nor was it in your initial code. I'm fine with adding the object info to the Symbol struct. However, if your plan is to load/save CW symbol maps in a strict manner, then you should provide the appropriate functions instead of changing the existing ones. For instance, you can add in the GUI (the load/save dialog or a dedicated menu) another symbol map format support. |
Fine, I'll compromise. I would like to discuss things further though later. |
55ac242
to
aa95805
Compare
aa95805
to
1c4bfc3
Compare
@sepalani Any updates? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code LGTM, tested with some games and it seems to work properly.
I think that it'd be better to have at least someone else (with a more involved symbol map/debugger use) to test this PR to make sure that it doesn't break on their end. Maybe @TryTwo or @dreamsyntax?
I'll try with a few maps and see how it looks soon. |
It would be nice to warn somehow that existing maps will be formatted, for those who do group collab/diffing - it could get messy if they don't know. -800ab58c 00000030 800ab58c 0 HAS_Player::StartJumpCommand_AND_Player::TouchHangCommand(?Regular2PWorkingPulley?)_800ab58c_
+800ab58c 000030 800ab58c 0 HAS_Player::StartJumpCommand_AND_Player::TouchHangCommand(?Regular2PWorkingPulley?)_800ab58c_^M [x] Old map format loads fine, saves in new format Edit: If you saw my message about an issue with Spyro, disregard. I made a mistake in my diffing. |
Approved with the caveat that I cannot explicitly test the My other user made maps and built in maps work as expected. |
I don't have anything to test. All my personal symbols were switched over to an extra note-layer in my personal debugger builds, because I like to mark certain instructions on top of symbol marking the functions. |
Conforming to new format introduced in dolphin-emu/dolphin#13113
This PR adds code that checks for extra information following symbol names that is often present in CodeWarrior symbol maps, namely the names of the .a/.o files that symbol belongs to. If present, the symbol name string is truncated to remove the extra text from the symbol name.