@orivej, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edolstra, @corngood and @peti to be potential reviewers.

Looks good to me (though bash is not an ideal language for writing parsers...). It should be applied to the staging branch though.

Changed target branch to staging. I though of writing in another language, but it seemed to be difficult to integrate with utils.sh/cc-wrapper.sh/ld-wrapper.sh.

Looks good for me, too; I can only guess what performance would be, but probably better on average than forking a compiled parser.

IMHO performance is acceptable (faster than a subsequent compiler, and .rsp files are rarely used), although a compiled parser would beat it: you can perform a realistic test with

source utils.sh
seq -f argument%.0f 100 > test.rsp
expandResponseParams @test.rsp

On my machine this 1KB file is parsed in 0.13 seconds.

This seems to be affecting linking times of Haskell projects (which are being linked with gcc and as I understand always linked using response files - https://ghc.haskell.org/trac/ghc/ticket/8596). On a project that otherwise builds in under one minute (not including dependencies) this change adds over a minute of overhead. Would it be worth filing an issue, or would that kind of overhead not be considered substantial enough? Happy to provide an .rsp file I extracted during the build (36KB, ~ 500 lines).

Could you upload an .rsp on https://gist.github.com/ ?
I observed that by far the slowest operation in the parser is array extension: rsp+=("$arg"). Maybe there is a simple way to optimize it.

@pbogdan Could you run the timing experiment @orivej described above, using both the synthetic test and your own response file, and post the times?

@orivej thanks, uploaded a file to https://gist.github.com/pbogdan/9d6986bf931b58a70d75e14eb40ee8a1 @corngood will run those shortly

This patch reduces parse time to 12 seconds. I'm going to see if it can be improved even further:

     local s="$1"
     local state=0
     local arg=''
+    local len="${#s}"
 
-    for (( i=0; i<${#s}; i++ )); do
+    for (( i=0; i<$len; i++ )); do
         local c="${s:$i:1}"
         local cls=1
         case "$c" in

#26554 parses your example in one second.