diff --git a/README.md b/README.md index f9db8ce..fb51563 100644 --- a/README.md +++ b/README.md @@ -12,10 +12,11 @@ A CLI tool for extracting and modifying text in .scx and .msb scripts found in v - STEINS;GATE 0 (Steam) - CHAOS;CHILD Love Chu☆Chu!! (PS4 & Impacto) - ROBOTICS;NOTES DaSH +- 11eyes CrossOver (Xbox 360) ## Usage -Run `./sc3tools` with no arguments to see the list of the avaliable commands, as well as the list of the supported games and their aliases (such as `sg0` for Steins;Gate 0). +Run `./sc3tools` with no arguments to see the list of the available commands, as well as the list of the supported games and their aliases (such as `sg0` for Steins;Gate 0). Run `./sc3tools help ` to see the help message for a specific command. @@ -23,4 +24,43 @@ Here's an example of how you can extract text from the Robotics;Notes scripts: `./sc3tools extract-text C:/src/CoZ/rne-msb/*.msb rn` +To use the `replace-text` function, the replacement `.txt` file has to be named in the same way as the original script file. + +It is also advised to create a copy of the original before replacement (copy source file, paste, then append `.org` to the end, so you know which is the **or**i**g**inal). + +If you've made sure to create a copy of the original, just use `sc3tools replace-text SC000.scr SC000.txt 11eyes` (in the case of 11eyes CrossOver) + The output files will be placed in a subfolder named `txt` (in this case, `C:/src/CoZ/rne-msb/txt`). + +## Compilation +Install the Rust toolchain for Windows from [rustup.rs](https://rustup.rs). + +Clone the repository using Git: `git clone https://github.com/ThePlayer14/sc3tools_mod.git` + +Navigate to the cloned folder `sc3tools_mod` and open the context menu / rightclick menu in File Explorer and click on "Open in Terminal" + +From this point you can run `cargo build` to build a "dev" (debug) release, or run `cargo build --release` to make a "release" build. + +## Features added alongside the color bugfix +**1. External game `resources` directory support** (`gamedef.rs` + `lib.rs`): +- New `--resources-dir ` CLI flag (global, works on both subcommands) +- `gamedef::load_resource()` — checks external path first, falls back to embedded +- `gamedef::load_gamedefs_json()` — loads from external file or embedded +- `gamedef::build_gamedefs_from_json_with_base()` — accepts external base path +- `GameDef::new()` now takes `external_base: Option<&Path>` +- Adding a new game = drop a folder with `charset.utf8` + `compound_chars.map` into the resources dir and add an entry to `gamedefs.json` — no code changes made + +**2. MagesTools-style tags in extraction** +- New `--mgs-format` CLI flag (use with `extract-text`) +- Outputs a MagesTools-compatible script from a compiled script file + +## Known issues +- This tool originally couldn't handle color setting in dialogue correctly (such as in the case of 11eyes CrossOver), and it will leave a truncated script if that is happened. + + Example of the telltale sign: +``` + Processing "D:\\script\\SC000.scr"... + Error: SC000.scr, line 73: expected more input. +``` +- With the release in the Releases tab, this is no longer the case. You can also build a functioning program for other platforms from the code. +- For details of the fix, see [The bugfix report.](https://github.com/ThePlayer14/sc3tools_mod/blob/main/rust-bugfix.md) diff --git a/resources/eeco/charset.utf8 b/resources/eeco/charset.utf8 new file mode 100644 index 0000000..08c2d87 --- /dev/null +++ b/resources/eeco/charset.utf8 @@ -0,0 +1 @@ + 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_/:-０１２３４５６７８９ＡＢＣＤＥＦＧＨＩＪＫＬＭＮＯＰＱＲＳＴＵＶＷＸＹＺａｂｃｄｅｆｇｈｉｊｋｌｍｎｏｐｑｒｓｔｕｖｗｘｙｚ、。，．：；？！゛゜‘’“”（）〔〕［］｛｝〈〉《》「」『』【】・…～ー♪─ぁぃぅぇぉっゃゅょゎァィゥェォッャュョヮヵヶ①②③④⑤⑥⑦⑧⑨⑩⑪⑫⑬⑭％⑯―━⑲⑳０１２３４５６７８９ＡＢＣＤＥＦＧＨＩＪＫＬＭＮＯＰＱＲＳＴＵＶＷＸＹＺａｂｃｄｅｆｇｈｉｊｋｌｍｎｏｐｑｒｓｔｕｖｗｘｙｚ、。，．：；？！゛゜‘’“”（）〔〕［］｛｝〈〉《》「」『』【】・…～ー♪─ぁぃぅぇぉっゃゅょゎァィゥェォッャュョヮヵヶ①②③④⑤⑥⑦⑧⑨⑩⑪⑫⑬⑭％⑯―━⑲⑳あいうえおかがきぎくぐけげこごさざしじすずせぜそぞただちぢつづてでとどなにぬねのはばぱひびぴふぶぷへべぺほぼぽまみむめもやゆよらりるれろわゐゑをんアイウエオカガキギクグケゲコゴサザシジスズセゼソゾタダチヂツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモヤユヨラリルレロワヰヱヲンヴ☆験逸剥専婦妻鎖既蒸鶴雫吉翻弄簡接充停灯朽凝幽針射裸粘狙遥角飢紙粟志柱賭亡率罰膨朧幾脈摘潜泡被易璧描絵鐘凍懐第爛致座芋骨肢順霧靄堵具尋泊跳草豹勿嫁添慮機卓齢告衝湯暖冴掛晩挨拶孫婚砲鮭噌汁豆揚癒喫茶円則掲例夏絞砕鳥門析謂頂凄版舗至索殻絆午担編装紹百干渉垣潤惚猫瞥漢資旅斜警戒策範田募看憩囁柄賑寧勉慎菓薔薇糖漬憂操輩波災井抵撫誤依南挽訣候翌沿芝民脅酔沢宇鉄雪／島幅犬戯招召菜旨炊研躍狭濯茂是寿司潮概箇択奮漏嗚咽閃壮聖粉均鈴穴鋭謎露鞘憑俗釈藁冒承宿顎拍除嘲涼敏儀即鎮麻痺祀藍晶祭唱律陣颯五煮滴就札作柔延毎土労車痴咲花弁濃覆群膜進宅暮退繰字息詰緩安疲萎袖横優為宙否糸肩寝訳謝曇元仕母族扉揮総希屍耐術叶泳魚浅鉛路並隅棄陰償温槌幻盛輝支誘従未醜胃液治誓抗快渦唾濡叩洗姉端頬伝菊鮮刀刃貼毛玉藻守秒魅芻慣精留要試臆廃拒助弟孤紛運暴医薬品睡負環輪呂厚寸玄惑客審奇危届団怒痒掻坊康参脱遅朝浴腰納件根施設陥養整準戦迫那鬱杯親境拠塵申弾★漕鬼闘台乗挑勝捨詞誇狗約束店敵国標因淡板共雑殿観氷低休叱袋箱量較繕売補暇遭辞令卑妖艶漂妹選据舌唇険捕睨呑児密富万牙熟唸超漫才揃鹿訪伏床娘戸忌擦権利酷潔況攻縁尻師代山鶏卵飯緑料修隙飾包丁膚触伸載敗器股席些激訝管援頑極錯縮堪蘇岸畑勘省兄筋捉倍増抑偶積己枕噛趣縛木霊裾湿歴史古橋働挟商画都釣徴工稼溶企奢詳塩油営拳震緒模腫級提眉蔵鵜透換棒蹴緊怯勇領域溢桃源占皇帝星争秘霞華舞公垂究越愉悦敷嬉圧溺窒襲鼓速移虫裕韻噴疑朱偏執遂暗西把雷陸居貸耗顔覗水奈瀬馴澄瞳防備動純粋年綺麗四六時少置配本当黙屈託持入禁随経閉議登好徐短晴通館市最組式称長普段呼由話忘般務授受室各階割廊反対側社科視聴覚文系活職用使図書健購買収故忍状態改問答困首産的彷徨説難待適読曲傷嫌嘘表情落線瞬微計差薄尖方新示印恥平照可寄評容姿周多功噂素歳相応雰囲恋早肌寒清邪旦戻珍隔仲悶番騒初冷汗拭外吐演技悔宝映像展帯写送電撮似内法程済効決常複忙位類盗駄十勢認題起基憎察替構離辺連期判丸投互机関付際軽鈍倒荷直撃香央里拾欠片礼暑単語細刷矢継発歯衣節型二性格頼友達役句殴貴過繊猿褒加減飽論枚責任襟掴偉非君爆嵐穏遣窓歩急寂訴係苛咥驚皿揉匂広丈夫影派羽織象個傍若介昔毒羨余眼他怪我病半黄症種障害距回左仮質久頃妙扱揺髪案腕抱甘酸呟抜腹甲斐鼻拡散嗅童貞処尾濁冗談殊慢能癖八九衆点慌限深欲終両避爽繋懲指定赤黒染逆巻紅蓮炎地舐異臭放煙天突死闇埋尽世光景到底私信一切救場所獄今確刺人肉布引千面血雨降注肝脳塗大軍馬身兵士何躊躇踏腐汚泥足纏目見昨日明笑隣物言塊化転思折重形丘者更気間尊厳与生意荒野晒獣骸同以下存在成考前臓腑絡太許這老体貫声上主祈捧絶命無識走出辛先逃来巨行遮街家火狂燃々焼焦知恐怖望交如叫果満界涙流続夢中途度壁打滑石畳取立止肘膝裏皮裂証辿導心教会向海込建払敬苦痛歪悲鳴潰彼嘆湧念自分飲泣再道塞育変様背豊美徒集遊詩歌滅刻迎略奪殺破壊悪跡灼熱消去予罪赦犯愛高御名口葉失返喉乾感崩手川浸誰握締張造原父偽威王慈真実事乱諦静沈雄業音轟断末魔遠男女幼子福呪住等歓喜有理唐悟数味牛食価値固溜呻願聞近剣胸解唯空仰東夜帳始夕陽混色灰帰哀魂眠怠隠神迷恩給白昼昇虚月課後屋独呆然眺俺青三別興秋渡雲次求蒼学園特耳傾球部員小金属響吹奏楽練習校舎彩吸現曖昧浮比儚弱残想砂不記憶押完全供違良幸必合得努力結報正義含悩調和保局堂巡着右奥追頭振開風強制服飛駆探槍襤褸鎧拉穢悼乞蹲婆痙攣辻篝虔怨嗟坩堝躓瓦礫貌吟穣虐徳痕嘯躙噎濘腸裁稜嘶祝戮誅弑殲鏖屠湛抉眩永燼牢惰愚寵仄棘挫疼帽鍵虹陵綾洋棟庭兎庫週淀奴筈匡懊携胞典賢蛮套諸紫燻嶺貧厄皐便併剰俯喰坂菖蒲季梅胡駅漠護芽搾窪嘔瀉剃■蝕褪弛涯錠郵旺喪櫛鏡勤憊駐∵篠朗貰遇粧彙某祟媚醗酵春籠抹費炒朴苔婿皆罵咀嚼妨循疎縫葦河植区層築貪粗嗜卒凛浪猟蠢嬌郷侵剛毅昏凡脊髄咄昆竦稀墨逡測弘睦旧姓墟郭膿忽禍糞捻煽澹推踊硝陶蛭維縷懺爬蓋跨斎刊醸諭衾乙孔堕曜賃恵征財篭村貶銘哲憤創膏愕鱗仏栞暢勧辟瘴紳洒杞衛巷条稚芦雇淹焙煎械汝犠牲軒迂痩北壇桜瓶彿梃肯羞慶戚馳芸燥稲漆斬帛戟餓脆穿畏只洪魄項騎鋼塔廉伽躯縋厭閇敢慄冑涜喩顕謐斂胚荘竜絹焔晦袂括褄仁侠倫診院煩宮虎郊疇稽廷裔衰章遼融籍排斥蹟還髭涛鑑茫瀕遍媛坦云僚喝橘輔傑墓拷徘徊媒仔誠削靴杓棺栖副詫綻兆婉盾薙療牡籥闔艸阜零儔烏勅吾伊松詠諳蝗螺鞭隊奉亨鼬蔽捩冠憐暈枷患城澱晰縊珈琲蟹閥帥郎祖楚魁杉党屁伐坐錬庁悉呵綱乳褐雅喚耽摺詛偵咎畜誕諺弦慕博覧牽ゝ請壷仙沌扇傲遜顧倣蜜姑謙呈謳碑侍塾叔京薦架倉朋楯亮酬闊臥沼契辣峙柳丹拮亀挺捲僥倖鎚鉈撲腱揶揄獅咆哮暫猶疾雁搦楔羅壱渾弐軸祓翔痣佇漲惧梁跋扈俊僧○恍侮旬撒獰恫遁隈錆蓑宜銃鍔臍尺磨炉叡智膣殖謬饒宗累匿雛阿弥陳滓燕囚蛾捌嘴抽顛酪農柿曹庶岳峰麓峠径浦桓鴉鉋鰹江県卍囃孕愁俵蝶邸欒隆掌釜禄囮綯刈椎軟贖嬲擲癇仇鎬肺蝋燭窩茸尤昂遺馨祉覯核著丼寡轢箒恭蘭綴賜暦芒廻搬俄磋琢罷践酌沙汰諍寓姦尚皺謗猥枠町泰淵駿駕填棲飴鏝燈餞榴眸帷坤堰恃疱涅緋傀儡糧瞑隷詣邦欧杭塚侯殆薀勒菩薩寺跏惟盞禅秦膾熨斗姜闖糾鋒熾唆＠胎曼荼蕩槃吶喊崎畿槽阪幌寛矯嫐杵勃綿嘗蜘蛛巣襞妊履佳篤傭縄杖之弔讐沫峯攪拌喀刮逮砦墾昭滲靭嚥埒驕闢恒吽轍逗柑柚撹殉霰嵩撥杏幡蟲宵瞭緻卯詈憔悴暁蒐遡浄盲拓廠盟踪躾簿竄瑣益訥鑽敦芳翠欺瞞秤述紡藤尿泌涎侭窄腋滾邂逅肥焉獪恰玩埃隼臣飼梢蕾梱腿伯磁葵擽逐蟻肛腺迸甦剖莫閨頚劈煉炙鍋藩孝稿貨糊甚翅啜賓勁詭鮫濛摯咫剋懣啖諌愾緞辜髑髏絢賊刳醇訛蒔屹猊蹂傅簒陛弊鞠鼎贋啓慟哭帆叢霜逝慙愧癸亥葡萄矜脛駻偈巓需箔氾濫堤鳳鋏鑿褻姪眈痍睥鬨禿筵粥逞縞●鍛翼吊武唖岩及阻粛脇号謀了悠検査嬢借損伴規政鷹谷寮統轄没栄掃漁譲紐猛厨房撤靡賞惨討英爪材製宣茎烈額船善沸騰徹氏瓜慰協拘彫筆妥瑞拗擁辱匹委往復宴鞄冊桶控妄塀林鉢餌競刑喧蛇七斉旋億凶封芯軌奔官桁兼硬汲串浜蓄採臨免渋蛙胆幕嚇算醒伺紀幹劣渇鷲符双掘拝曰須憧騙謹軋懸酒缶励葛舟促詮盤崖刹挙樹肋鳩×煌胴咳懇墜劇網龍粒克訓緯箸賛講蔑販津朦餅葬紺薫狩惜竹礎革泉預碍傘釘狼摂秀惹銀将勾笛米摩脂＆航縦圏萌堅又享頻繁佃罠恨獲凌弧鎌豚姫喘窮岬録績垢贈椅擬列却棚＝矛赴苗培献頓筒衡僅炸枝劫嫉妬州府穫慨枯袈裟乏催窺披豪巾閑腔竿挿淫覇誉脚監膳紋碗忠池炭臼佐弓吼枢滞蚊羊焚踵贅戴掬虜崇冬珠剤岐吠票欄滝冥迦森洞僕遽駒贄掠狸雌耕捜巫銭－頷貢棍饅呷縺訊娠跪噺屑Ⅱ倦娶芥汐淑澪睫巧噪靱爾諾淘播狡猾鼠閲嘩而頁批汎諜秩序熊楠圻杣孵苺庸賎遙輸飄衒該瑕磔斧迅蜂檻庇奸鵰絨毯癪喋盆譜熔梨牒疫　 \ No newline at end of file diff --git a/resources/eeco/compound_chars.map b/resources/eeco/compound_chars.map new file mode 100644 index 0000000..e69de29 diff --git a/resources/gamedefs.json b/resources/gamedefs.json index 4fcae08..951498a 100644 --- a/resources/gamedefs.json +++ b/resources/gamedefs.json @@ -70,5 +70,11 @@ "resource_dir": "ia", "aliases": ["ia", "iwakuraaria"], "fullwidth_blocklist": ["'"] + }, + { + "name": "11eyes CrossOver Xbox 360", + "resource_dir": "eeco", + "aliases": ["11eyes", "eleveneyes"], + "fullwidth_blocklist": ["'"] } -] \ No newline at end of file +] diff --git a/rust-bugfix.md b/rust-bugfix.md new file mode 100644 index 0000000..c08ea1d --- /dev/null +++ b/rust-bugfix.md @@ -0,0 +1,32 @@ +# What the Fix Was + +The fix addresses a fundamental misunderstanding of how the **SetColor (0x04)** command's data length is encoded in SC3 scripts. The Rust-based `sc3tools` incorrectly handled color data: + +**The Bug:** The original code assumed SetColor always uses either: +- A simple 1-byte value (X360 games), or +- A full "expression" with a 0x00 terminator (Steam games) + +The Rust `sc3tools` (`sc3tools/src/sc3.rs`) parses SetColor using `Expr::parse`, which treats the color data as a generic expression terminated by `0x00`. This **double-counts the 0x00 terminator**, causing "expected more input" errors. The older summary (part of a previous run) explicitly states: *"Rust's `Expr::parse` is independently broken (double-counts the `0x00` terminator → 'expected more input')."* + +### The Rust sc3tools Bug (fixed) +In `sc3tools/src/sc3.rs` (Line 149): +```rust +0x04 => parse(i, Expr::parse, StringToken::Color), +``` + +And `Expr::parse` (Lines 95-99): +```rust +pub fn parse(i: &'a [u8]) -> IResult<&'a [u8], Self> { + map(recognize(many_till(Self::token, tag(&[0x00u8]))), |slice| { + Expr(Cow::from(slice)) + })(i) +} +``` + +The `many_till(Self::token, tag(&[0x00u8]))` reads tokens until it finds a `0x00` byte, but the `0x00` terminator is included in the recognized slice. The `Expr::parse` then expects more input after the terminator (since `recognize` consumes it but `many_till` stops at it without consuming), causing the "expected more input" error. + +### The implemented fix: +**SetColor color fix** (`sc3.rs`): +- `StringToken::Color` now stores `Cow<'a, [u8]>` (raw bytes) instead of `Expr<'a>` +- New `parse_color` function: reads first byte, if `< 0x80` → 1 byte (X360), if `>= 0x80` → `const_len + 1` bytes including terminator (Steam) +- Updated `coz.rs` serialize/deserialize to handle raw bytes diff --git a/src/coz.rs b/src/coz.rs index dae293b..24d5604 100644 --- a/src/coz.rs +++ b/src/coz.rs @@ -90,6 +90,62 @@ impl<'a> Sc3String<'_> { Ok(CozString(Cow::from(buf))) } + pub fn serialize_mgs(&self, gamedef: &GameDef) -> Result { + fn hex2(val: u16) -> String { + format!("{:02X}{:02X}", (val >> 8) as u8, val as u8) + } + + let mut buf = String::new(); + for tk in self.iter() { + let tk = tk?; + match tk { + sc3::StringToken::Text(encoded_text) => { + let s = text::decode_str_with_options(&encoded_text, &gamedef, true, true)?; + write!(buf, "{}", s.as_str()).unwrap(); + } + sc3::StringToken::LineBreak => write!(buf, "[0x00]").unwrap(), + sc3::StringToken::NameStart => write!(buf, ":[").unwrap(), + sc3::StringToken::LineStart => write!(buf, "]:").unwrap(), + sc3::StringToken::Present(action) => match action { + sc3::PresentAction::None => write!(buf, "[0x03]").unwrap(), + sc3::PresentAction::ResetAlignment => write!(buf, "[0x08]").unwrap(), + sc3::PresentAction::Unknown_0x05 => write!(buf, "[0x05]").unwrap(), + sc3::PresentAction::Unknown_0x18 => write!(buf, "[0x18]").unwrap(), + }, + sc3::StringToken::Color(color_value) => { + write!(buf, "[0x04").unwrap(); + for b in color_value.iter() { + write!(buf, "{:02X}", b).unwrap(); + } + write!(buf, "]").unwrap(); + } + sc3::StringToken::RubyBaseStart => write!(buf, "[0x09]").unwrap(), + sc3::StringToken::RubyTextStart => write!(buf, "[0x0A]").unwrap(), + sc3::StringToken::RubyTextEnd => write!(buf, "[0x0B]").unwrap(), + sc3::StringToken::FontSize(val) => write!(buf, "[0x0C{}]", hex2(val)).unwrap(), + sc3::StringToken::Parallel => write!(buf, "[0x0E]").unwrap(), + sc3::StringToken::Center => write!(buf, "[0x0F]").unwrap(), + sc3::StringToken::MarginTop(val) => write!(buf, "[0x11{}]", hex2(val)).unwrap(), + sc3::StringToken::MarginLeft(val) => write!(buf, "[0x12{}]", hex2(val)).unwrap(), + sc3::StringToken::HardcodedValue(val) => write!(buf, "[0x13{}]", hex2(val)).unwrap(), + sc3::StringToken::Eval(expr) => { + write!(buf, "[0x15").unwrap(); + for b in expr.0.iter() { + write!(buf, "{:02X}", b).unwrap(); + } + write!(buf, "]").unwrap(); + } + sc3::StringToken::AutoForward => write!(buf, "[0x19]").unwrap(), + sc3::StringToken::AutoForward_1A => write!(buf, "[0x1A]").unwrap(), + sc3::StringToken::RubyCenterPerChar => write!(buf, "[0x1E]").unwrap(), + sc3::StringToken::AltLineBreak => write!(buf, "[0x1F]").unwrap(), + sc3::StringToken::Terminator => {} + } + } + write!(buf, "[0xFF]").unwrap(); + Ok(CozString(Cow::from(buf))) + } + pub fn deserialize( s: &'a CozString, gamedef: &GameDef, @@ -221,7 +277,7 @@ impl<'a> StringToken<'a> { StringToken::MarginLeft(val) => ("margin", Some(("left", val.to_string()))), StringToken::MarginTop(val) => ("margin", Some(("top", val.to_string()))), StringToken::Terminator => ("", None), - StringToken::Color(expr) => ("color", Some(("index", hex::encode_upper(&expr.0)))), + StringToken::Color(color_value) => ("color", Some(("index", hex::encode_upper(&color_value)))), StringToken::FontSize(val) => ("font", Some(("size", val.to_string()))), StringToken::HardcodedValue(val) => { ("hardcoded-value", Some(("index", val.to_string()))) @@ -261,7 +317,7 @@ impl<'a> StringToken<'a> { "%e" => Ok(StringToken::Present(sc3::PresentAction::ResetAlignment)), "%05" => Ok(StringToken::Present(sc3::PresentAction::Unknown_0x05)), "%18" => Ok(StringToken::Present(sc3::PresentAction::Unknown_0x18)), - "color" => Self::expr_attr(tag.attr.as_ref(), "index").map(StringToken::Color), + "color" => Self::bytes_attr(tag.attr.as_ref(), "index").map(StringToken::Color), "ruby-base" | "rubybase" => Ok(StringToken::RubyBaseStart), "ruby-text-start" | "rubytextstart" => Ok(StringToken::RubyTextStart), "ruby-text-end" | "rubytextend" => Ok(StringToken::RubyTextEnd), @@ -337,6 +393,20 @@ impl<'a> StringToken<'a> { Self::get_attr(attr, name, expr) } + + fn bytes_attr( + attr: Option<&Attr<'a>>, + name: &'a str, + ) -> Result, ParseError> { + fn hex_string(value: &str) -> IResult<&str, Vec> { + many1(map_res( + take_while_m_n(2, 2, |c: char| c.is_digit(16)), + |hex| u8::from_str_radix(hex, 16), + ))(value) + } + + Self::get_attr(attr, name, hex_string).map(Cow::from) + } } impl From for Error { diff --git a/src/gamedef.rs b/src/gamedef.rs index c92980c..f22f3d5 100644 --- a/src/gamedef.rs +++ b/src/gamedef.rs @@ -9,7 +9,7 @@ use nom::{ IResult, }; use rust_embed::RustEmbed; -use std::{borrow::Cow, collections::HashMap, ops::RangeInclusive}; +use std::{borrow::Cow, collections::HashMap, ops::RangeInclusive, path::Path}; use serde::Deserialize; use serde_json; @@ -17,6 +17,20 @@ use serde_json; #[folder = "resources/"] pub struct ResourceDir; +fn load_resource(base_path: Option<&Path>, resource_dir: &str, file_name: &str) -> Cow<'static, [u8]> { + if let Some(base) = base_path { + let path = base.join(resource_dir).join(file_name); + if path.exists() { + return Cow::Owned(std::fs::read(&path).unwrap_or_else(|e| { + panic!("Failed to read resource file {:?}: {}", path, e) + })); + } + } + let embedded_path = format!("{}/{}", resource_dir, file_name); + ResourceDir::get(&embedded_path) + .unwrap_or_else(|| panic!("Resource not found: {} (also checked external path: {:?})", embedded_path, base_path)) +} + pub struct GameDef { #[allow(dead_code)] pub full_name: String, @@ -47,6 +61,7 @@ impl<'a> From> for GameDef { json.aliases, json.reserved_codepoints, json.fullwidth_blocklist, + None, ) } } @@ -58,19 +73,16 @@ impl GameDef { aliases: Vec, reserved_codepoints: Option>, fullwidth_blocklist: Vec, + external_base: Option<&Path>, ) -> Self { - fn file_path(resource_dir: &str, name: &'static str) -> String { - format!("{}/{}", resource_dir, name) - } - let charset: Cow<[u8]> = - ResourceDir::get(&file_path(resource_dir, "charset.utf8")).unwrap(); + load_resource(external_base, resource_dir, "charset.utf8"); let charset: Vec = std::str::from_utf8(charset.as_ref()) .unwrap() .chars() .collect(); let compound_chars: Cow<[u8]> = - ResourceDir::get(&file_path(resource_dir, "compound_chars.map")).unwrap(); + load_resource(external_base, resource_dir, "compound_chars.map"); let compound_chars = std::str::from_utf8(compound_chars.as_ref()).unwrap(); let compound_chars = parse_compound_ch_map(compound_chars); let encoding_maps = EncodingMaps::new(&charset, &compound_chars); @@ -98,6 +110,17 @@ impl GameDef { } } + pub fn from_with_base(json: GameDefJson<'_>, external_base: Option<&Path>) -> Self { + Self::new( + json.name, + json.resource_dir, + json.aliases, + json.reserved_codepoints, + json.fullwidth_blocklist, + external_base, + ) + } + pub fn charset(&self) -> &[char] { &self.charset } @@ -113,6 +136,24 @@ pub fn build_gamedefs_from_json(json: &str) -> Vec { defs.into_iter().map(GameDef::from).collect() } +pub fn build_gamedefs_from_json_with_base(json: &str, external_base: Option<&Path>) -> Vec { + let defs: Vec = serde_json::from_str(json).unwrap(); + defs.into_iter().map(|d| GameDef::from_with_base(d, external_base)).collect() +} + +pub fn load_gamedefs_json(external_base: Option<&Path>) -> String { + match external_base { + Some(base) => std::fs::read_to_string(base.join("gamedefs.json")) + .unwrap_or_else(|e| panic!("Failed to read gamedefs.json from {:?}: {}", base, e)), + None => { + let file = ResourceDir::get("gamedefs.json") + .expect("Embedded gamedefs.json not found"); + String::from_utf8(file.to_vec()) + .expect("gamedefs.json is not valid UTF-8") + } + } +} + #[derive(Eq, PartialEq, Debug)] struct PuaMapping<'a> { codepoint_range: RangeInclusive, diff --git a/src/lib.rs b/src/lib.rs index 4648037..040ff27 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -26,12 +26,30 @@ use std::{ fs::{self, OpenOptions}, io, io::{BufRead, BufReader, BufWriter, Write}, - path::PathBuf, + path::{Path, PathBuf}, }; -use std::{error::Error, fs::File, path::Path}; +use std::{error::Error, fs::File}; use termcolor::{Color, ColorChoice, ColorSpec, StandardStream, WriteColor}; +fn detect_external_resources() -> Option { + let args: Vec = std::env::args().collect(); + for (i, arg) in args.iter().enumerate() { + if arg == "--resources-dir" { + if let Some(path) = args.get(i + 1) { + let pb = PathBuf::from(path); + if !pb.exists() { + eprintln!("Warning: --resources-dir path does not exist: {:?}", pb); + } + return Some(pb); + } else { + eprintln!("Warning: --resources-dir requires a path argument"); + } + } + } + None +} + #[derive(Debug)] enum ProcessingError { Script(PathBuf, usize, Box), @@ -43,10 +61,10 @@ enum ProcessingError { impl error::Error for ProcessingError {} pub fn run() -> Result<(), Box> { - - let game_defs_file = gamedef::ResourceDir::get("gamedefs.json").unwrap(); - let game_defs_json = std::str::from_utf8(game_defs_file.as_ref()).unwrap(); - let defs = gamedef::build_gamedefs_from_json(game_defs_json); + + let external_resources = detect_external_resources(); + let game_defs_json = gamedef::load_gamedefs_json(external_resources.as_deref()); + let defs = gamedef::build_gamedefs_from_json_with_base(&game_defs_json, external_resources.as_deref()); let supported_games: Vec = defs.iter() .flat_map(|v| v.aliases.iter().cloned()) // Clone the strings to own them .collect(); @@ -77,6 +95,13 @@ pub fn run() -> Result<(), Box> { .author("Committee of Zero") .version("2.1") .after_help(&after_help) + .arg( + Arg::new("resources-dir") + .long("resources-dir") + .help("Path to external game resources directory (contains gamedefs.json and game subdirs)") + .required(false) + .global(true), + ) .subcommand( Command::new("extract-text") .about("Extracts text from one or multiple script files") @@ -92,7 +117,13 @@ pub fn run() -> Result<(), Box> { .long("preserve-fullwidth") .action(ArgAction::SetTrue) .help("Preserve fullwidth characters") + .required(false), + Arg::new("mgs-format") + .long("mgs-format") + .action(ArgAction::SetTrue) + .help("Output in MAGES-compatible raw byte format ([0x..] tags, raw compound chars, [0xFF] terminator)") .required(false) + .overrides_with("preserve-fullwidth"), ]), ) .subcommand( @@ -125,7 +156,8 @@ pub fn run() -> Result<(), Box> { let game = sub_m.get_one::("game").unwrap(); let gamedef = gamedef::get_by_alias(&defs, game).unwrap(); let keep_fullwidth_chars = sub_m.get_flag("preserve-fullwidth"); - run_extract_text(parse_glob("input", input)?, gamedef, keep_fullwidth_chars) + let mgs_format = sub_m.get_flag("mgs-format"); + run_extract_text(parse_glob("input", input)?, gamedef, keep_fullwidth_chars, mgs_format) } Some(("replace-text", sub_m)) => { let scripts = sub_m.get_one::("scripts").unwrap(); @@ -148,6 +180,7 @@ fn run_extract_text( paths: Paths, gamedef: &GameDef, keep_fullwidth_chars: bool, + mgs_format: bool, ) -> Result<(), Box> { Ok(for entry in paths { let path = entry?; @@ -168,7 +201,7 @@ fn run_extract_text( println!("Processing {:?}...", path); let ext = ".".to_owned() + &path.extension().unwrap_or_default().to_str().unwrap() + ".txt"; let output = out_dir.join(stem + &ext); - if let Err(err) = extract_text(&path, &output, gamedef, keep_fullwidth_chars) { + if let Err(err) = extract_text(&path, &output, gamedef, keep_fullwidth_chars, mgs_format) { report_err(err) } }) @@ -203,6 +236,7 @@ fn extract_text( out: &impl AsRef, gamedef: &GameDef, keep_fullwidth_chars: bool, + mgs_format: bool, ) -> Result<(), Box> { let script = format::open(File::open(script_path)?)?; let txt = File::create(out)?; @@ -211,11 +245,13 @@ fn extract_text( let table = &script.string_index(); for (i, handle) in table.iter().enumerate() { let line = script.read_string(handle)?; - let serialized = line - .serialize(&gamedef, keep_fullwidth_chars) - .map_err(|err| { - ProcessingError::Script(script_path.as_ref().to_owned(), i, Box::new(err)) - })?; + let serialized = if mgs_format { + line.serialize_mgs(&gamedef) + .map_err(|err| ProcessingError::Script(script_path.as_ref().to_owned(), i, Box::new(err)))? + } else { + line.serialize(&gamedef, keep_fullwidth_chars) + .map_err(|err| ProcessingError::Script(script_path.as_ref().to_owned(), i, Box::new(err)))? + }; writeln!(writer, "{}", serialized)?; } diff --git a/src/sc3.rs b/src/sc3.rs index dc770b7..77fe43f 100644 --- a/src/sc3.rs +++ b/src/sc3.rs @@ -59,7 +59,7 @@ pub enum StringToken<'a> { NameStart, LineStart, Present(PresentAction), - Color(Expr<'a>), + Color(Cow<'a, [u8]>), RubyBaseStart, RubyTextStart, RubyTextEnd, @@ -117,6 +117,21 @@ impl<'a> Expr<'a> { } } +fn parse_color<'a>(i: &'a [u8]) -> IResult<&'a [u8], StringToken<'a>> { + let (i, first) = be_u8(i)?; + let (i, rest) = if first >= 0x80 { + let num_const_bytes = ((first & 0xE0).saturating_sub(0x80)) / 0x20 + 1; + let total_rest = num_const_bytes + 1; + take(total_rest as usize)(i)? + } else { + (i, &[] as &[u8]) + }; + let mut bytes = Vec::with_capacity(1 + rest.len()); + bytes.push(first); + bytes.extend_from_slice(rest); + Ok((i, StringToken::Color(Cow::from(bytes)))) +} + impl<'a> StringToken<'_> { pub fn decode(i: &[u8]) -> Result<(&[u8], StringToken), Error> { fn parse<'a, O, P, F>(i: &'a [u8], parser: P, f: F) -> Result<(&[u8], StringToken), Error> @@ -146,7 +161,9 @@ impl<'a> StringToken<'_> { 0x01 => Ok((i, StringToken::NameStart)), 0x02 => Ok((i, StringToken::LineStart)), 0x03 => Ok((i, StringToken::Present(PresentAction::None))), - 0x04 => parse(i, Expr::parse, StringToken::Color), + 0x04 => { + Ok(parse_color(i).map_err(|_| Error::ExpectedMoreInput)?) + } 0x05 => Ok((i, StringToken::Present(PresentAction::Unknown_0x05))), 0x08 => Ok((i, StringToken::Present(PresentAction::ResetAlignment))), 0x09 => Ok((i, StringToken::RubyBaseStart)), @@ -209,7 +226,7 @@ impl<'a> StringToken<'_> { sink.write(&code.to_be_bytes()).map(|_| ())?; match self { - StringToken::Color(expr) => sink.write(&expr.0), + StringToken::Color(bytes) => sink.write(&bytes), StringToken::FontSize(val) => sink.write(&val.to_be_bytes()), StringToken::MarginTop(val) => sink.write(&val.to_be_bytes()), StringToken::MarginLeft(val) => sink.write(&val.to_be_bytes()), @@ -254,4 +271,40 @@ mod tests { let expr = vec![0x29, 0x0A, 0xA0, 0x5A, 0x14, 0x14, 0x00, 0x80, 0x00, 0x00]; assert_eq!(Expr::parse(&expr).unwrap().1, Expr(Cow::from(&expr))); } + + #[test] + fn parse_color_x360() { + let data = vec![0x04u8, 0x02]; + let res = StringToken::decode(&data); + assert!(res.is_ok()); + let (rem, tk) = res.unwrap(); + assert_eq!(rem.len(), 0); + assert_eq!(tk, StringToken::Color(Cow::from(&[0x02u8][..]))); + } + + #[test] + fn parse_color_steam() { + let data = vec![0x04u8, 0x82, 0x00, 0x00]; + let res = StringToken::decode(&data); + assert!(res.is_ok()); + let (rem, tk) = res.unwrap(); + assert_eq!(rem.len(), 0); + assert_eq!(tk, StringToken::Color(Cow::from(&[0x82u8, 0x00, 0x00][..]))); + } + + #[test] + fn color_roundtrip_x360() { + let orig = StringToken::Color(Cow::from(&[0x02u8][..])); + let mut buf = Vec::new(); + orig.encode(&mut buf).unwrap(); + assert_eq!(buf, vec![0x04, 0x02]); + } + + #[test] + fn color_roundtrip_steam() { + let orig = StringToken::Color(Cow::from(&[0x82u8, 0x00, 0x00][..])); + let mut buf = Vec::new(); + orig.encode(&mut buf).unwrap(); + assert_eq!(buf, vec![0x04, 0x82, 0x00, 0x00]); + } } diff --git a/src/text.rs b/src/text.rs index 9a5c5fc..1107d41 100644 --- a/src/text.rs +++ b/src/text.rs @@ -183,10 +183,19 @@ pub fn decode_str<'a>( s: &[u16], gamedef: &'a GameDef, keep_fullwidth_chars: bool, +) -> Result, EncodingError> { + decode_str_with_options(s, gamedef, keep_fullwidth_chars, false) +} + +pub fn decode_str_with_options<'a>( + s: &[u16], + gamedef: &'a GameDef, + keep_fullwidth_chars: bool, + preserve_private_use: bool, ) -> Result, EncodingError> { let chars = s .iter() - .map(|code| decode_char(*code, gamedef.charset(), &gamedef.compound_chars)) + .map(|code| decode_char_with_options(*code, gamedef.charset(), &gamedef.compound_chars, preserve_private_use)) .collect::, _>>()?; Ok(Text::from_chars(chars.into_iter(), keep_fullwidth_chars)) } @@ -195,6 +204,15 @@ pub fn decode_char<'a>( code: u16, charset: &[char], compound_map: &'a HashMap, +) -> Result, EncodingError> { + decode_char_with_options(code, charset, compound_map, false) +} + +fn decode_char_with_options<'a>( + code: u16, + charset: &[char], + compound_map: &'a HashMap, + preserve_private_use: bool, ) -> Result, EncodingError> { let i = (code & 0x7FFF) as usize; let ch = charset @@ -203,10 +221,14 @@ pub fn decode_char<'a>( .ok_or_else(|| EncodingError::IllegalCharCode(code))?; if let '\u{e000}'..='\u{f8ff}' = ch { // Private Use Area - compound_map - .get(&ch) - .map(|s| Char::Compound(s)) - .ok_or_else(|| EncodingError::PuaCharNotMapped(code, ch)) + if preserve_private_use { + Ok(Char::Regular(ch)) + } else { + compound_map + .get(&ch) + .map(|s| Char::Compound(s)) + .ok_or_else(|| EncodingError::PuaCharNotMapped(code, ch)) + } } else { Ok(Char::Regular(ch)) }