@nikias, In iOS 13, most of the services require an extra SSL handshake once the service socket is connected (not just for the startService in lockdown client). I think this is the cause here too.

What do you mean by "extra SSL handshake"?
Can you please elaborate?

Lockdown return an extra key in response to StartService - EnableServiceSSL. When it is true, an SSL handshake is required on the new service socket after usbmuxd Connect.

@EliyahuStern but this is actually implemented here:
https://github.com/libimobiledevice/libimobiledevice/blob/master/src/lockdown.c#L1352
and here:
https://github.com/libimobiledevice/libimobiledevice/blob/master/src/service.c#L71
Is there anything else required?

@nikias you are right, missed that somehow. I will take another look...

I believe they switched to TLS 1.3 for ssl

@umutuzgur what makes you think that? I am able to successfully complete TLS handshake with protocol version 3.3 (TLS 1.2).

The Wired thing is, it seems like after the handshake is complete, some services (including debugserver) switch back to plain-text. No clue how to tell in runtime if SSL needs to be disabled or not after handshake though.

Are you able to read the service response by service_receive function on the SSL handshaked connection at all? (SSL or textual)
Because it seems like starting the point SSL handshake is successful on the service connection, the service response is not readable on the same connection (and yes, the response is sent textually by the device).

Yes, it looks like most services beyond the lockdown service require EnableSSL now. But once the handshake is performed, with the current code, the data is not encrypted. Some services do encrypt the but not all.
Could it be they use NULL cipher ? SLL with no encryption ?
Anyone capable of writing a new "idevice_error_t idevice_connection_enable_ssl(idevice_connection_t connection)" with NULL cipher to see if that is the problem ?

The service and client negotiate the cipher and if they don't agree the handshake would fail. So not sure how that would make sense. Do we have any proof/dump that shows that despite the successful handshake it would then continue in plain text? From my understanding it just stalls. Also which other services except debugserver don't work (after the most recent commits)? What services don't use SSL? From what I can tell all are forced to use SSL now.

That's really odd. I've looked at the Instruments service (com.apple.instruments.remoteserver). What I can see is that there now is a SSL handshake at the start (e.g. EnableSessionSSL, but SSL is stopped as soon as authentication succeeds), followed by plain text communication.

So apart from some services now requiring a SSL handshake where they apparently did not require one before, I don't see much of a difference.

@JonGabilondoAngulo You seem to be saying that you are observing services which require continued communication over SSL. Which services are that? Do you have any traces you could share?

I pushed a fix in 7c00b36.

I was working on "com.apple.instruments.remoteserver" as well, and as you say after the SLL_do_handshake the service returns plain text. I thought maybe it was some NULL cipher stuff. But I'll check nikias' last push it seems promising.
There is another service "com.apple.dt.fetchsymbols", which does the SSL handshake and the data is encrypted all along. No plain text on this one.
Different behaviours in ios13 services.

It looks like some services need the SSL_Handshake just for the start, then they go on without encryption. I guess some Apple teams did not get in time to iOS13 to make the communication encrypted, in the case of debugserver, being open source, maybe it will be not SSL for a while.

Anyhow I get better results in other services if instead of doing a "service_disable_ssl" I do force to read/write instead of SSL_read/write.

Implementing some kind of "service_bypass_ssl(parent)" ( instead of "service_disable_ssl(parent)" ), that will make "idevice_connection_receive_timeout" to jump to "internal_connection_receive_timeout" instead of doing the SSL transmission : "int r = SSL_read(connection->ssl_data->session, (void*)((char*)(data+received)), (int)len-received);"

I think something in this direction will make libimobiledevice more close to what really happens with the ios services.

LIBIMOBILEDEVICE_API idevice_error_t idevice_connection_receive_timeout(idevice_connection_t connection, char *data, uint32_t len, uint32_t *recv_bytes, unsigned int timeout)
{
	if (!connection || (connection->ssl_data && !connection->ssl_data->session) || len == 0) {
		return IDEVICE_E_INVALID_ARG;
	}

	if (connection->ssl_data   &&  !FORCED_TO_BYPASS_SSL ) {
		uint32_t received = 0;

		while (received < len) {

			int conn_error = socket_check_fd((int)(long)connection->data, FDM_READ, timeout);
			idevice_error_t error = socket_recv_to_idevice_error(conn_error, len, received);

			switch (error) {
				case IDEVICE_E_SUCCESS:
					break;
				case IDEVICE_E_UNKNOWN_ERROR:
					debug_info("ERROR: socket_check_fd returned %d (%s)", conn_error, strerror(-conn_error));
				default:
					return error;
			}

#ifdef HAVE_OPENSSL
			int r = SSL_read(connection->ssl_data->session, (void*)((char*)(data+received)), (int)len-received);
#else
			ssize_t r = gnutls_record_recv(connection->ssl_data->session, (void*)(data+received), (size_t)len-received);
#endif
			if (r > 0) {
				received += r;
			} else {
				break;
			}
		}

		debug_info("SSL_read %d, received %d", len, received);
		if (received < len) {
			*recv_bytes = 0;
			return IDEVICE_E_SSL_ERROR;
		}
		
		*recv_bytes = received;
		return IDEVICE_E_SUCCESS;
	}
	return internal_connection_receive_timeout(connection, data, len, recv_bytes, timeout);
}

Hey guys - I have been looking at the instruments service too <com.apple.instruments.remoteserver> after implementing some of the suggestions mentioned above by @JonGabilondoAngulo. I am getting strange results. Following are the steps I have taken.

if (IDEVICE_E_SUCCESS != idevice_new(&device, udid)) {
     printf("No device found, is it plugged in?\n");
     return -1;
}
if (LOCKDOWN_E_SUCCESS != (ldret = lockdownd_client_new_with_handshake(device, &lckd, "ideviceinstruments"))) {
        printf("ERROR: Could not connect to lockdown, error code %d.\n", ldret);
        goto leave;
}
 lockdownd_start_service(lckd, "com.apple.instruments.remoteserver", &service);
 // following function is similar to creation of a new client in image_mounter for instance
 // bypass SSL (dont disable as suggested above)
 if (instruments_client_new(device,NULL, 1, service, &instClient) != INSTRUMENTS_E_SUCCESS) {
        printf("ERROR: Could not connect to instrumets \n");
        goto leave;
 }
 // The following function is just a loop waiting to read for messages in the service.
 //Reads the first message fine.
//After that throws IDEVICE_E_TIMEOUT error when it comes to the second message.
instruments_receive_loop(instClient);

Have you proceeded beyond this @qmfrederik or @JonGabilondoAngulo ?

Also, are you guys using the latest commit from master or are you using a stable release (a.ka 1.2.0)?

Ok - upon doing some more investigation. It looks like I get to read the first message unencrypted without an issue.

After that, I get an SSL record(message) I think, the first three bytes are 0x15, 0x3, 0x1. And the next two bytes are 0x0 and 0x20. From the SSL Record format, it looks like the first byte denotes the type of message - 0x15 is an alert in this case. The next two bytes denote the version of SSL being used and the last two bytes are the size of the message. I am using release 1.2.0 of libimobiledevice and haven't updated to the latest code in the master branch.

So, one of the two could be happening, (speculation)

1. SSL Handshake has had an issue and its raising an alert from the device?

2. Alert is harmless but the next message is encrypted and we should be switching to read using SSL_read instead of bypassing?

Any ideas?

This sequence works:

• Start service with lockdownd_start_service, com.apple.instruments.remoteserver
• Do the connect via instrserver_client_new, SSL handshake is performed.
• Do the bypass
• read the the capabilities response (non SSL thank to bypass), arrived unencrypted.
• Open a channel ..
  etc.

If you stop or pause debugging at some step then you will get the SSL message you mention, probably some disconnection.

MESSAGE. Size:37
0000 15 03 01 00 20 98 ae bd f4 7d fb 1e 7d 58 55 42 .... ....}..}XUB
0010 3a 9a 96 da 61 24 73 6d f7 12 61 b8 28 51 7c 06 :...a$sm..a.(Q|.
0020 23 64 48 1e 5d #dH.]

@karthikgbbristol can you keep reading plaintext after receiving the alert?
I was debugging a similar case with "StopSession" message in lockdown. After sending the encrypted "StopSession" message, it seems like both sides sends a close_notify type of alert, which means the next message they send / expect is unencrypted.
I don't understand exactly how close_notify works, since one peer can send encrypted messages before receiving the close_notify alert from the other peer, so my guess is that the one initiating the close close should keep handling both encrypted AND plaintext messages until the other peer sends its close_notify response.
Maybe that's the case here too, and that's how Instruments knows to disable the ssl session, by receiving this alert. If that's right, maybe we should be inspecting every packet coming from the service to see if it matches SSL header (0x14/0x15/0x16/0x17 and 2 then version bytes, and two size bytes matching the size of this record).

This is of course a speculation, since the alert itself is encrypted and I can't tell its contents (could not reproduce it myself yet).

@JonGabilondoAngulo I am pretty sure that's the scenario I am using now. But after I send a message to open a channel, I get the SSL Alert which upon more investigation is a SSL_SHUTDOWN. I will try and read the 37 bytes of this message first to see if the host can proceed unencrypted as suggested by @EliyahuStern.
Thanks a bunch for your help on this. I will keep this issue updated with my findings.

I can go on to open a channel, ask for process id, launch apps .. no problem of disconnection.
Of course i use the latest code in libimobiledevice, libusbmuxd ...
Again, if i'm slow, I get the disconnection.

By the way this is another problem, I don't know if it related to what you see.
#807 (comment)

@JonGabilondoAngulo . Which particular service are you using?
I am trying the sequence you mentioned with com.apple.instruments.server.services.deviceinfo service. When I send the open channel request, the device responds with a SSL_SHUTDOWN message. I tried reading the SSL Alert and proceeding, but can't read anything else after that (getting ECONNRESET). So it looks like it shuts the whole connection down.

I will try using the latest version of libimobiledevice and libusbmuxd and recreate my test. I have tried the mod you suggested in #807 too. But that's an issue related to timeout and I don't think it helped.

I have updated to the latest code from libimobiledevice and libusbmuxd and still get the SSL SHUTDOWN.

Following is the sequence of calls I make (I have written a stub program that just reads the capability message using the bypass, then sends a simple command (another request for capabilities) and reads the response. When reading the response I always get a SSL SHUTDOWN message. Fails every time on my iPhone 6s Plus.

//Create new idevice
if (IDEVICE_E_SUCCESS != idevice_new(&device, udid)) {
    printf("No device found, is it plugged in?\n");
    return -1;
}
//Create lockdown client with handshake
 if (LOCKDOWN_E_SUCCESS != (ldret = lockdownd_client_new_with_handshake(device, &lckd, 
 "ideviceinstruments"))) {
    printf("ERROR: Could not connect to lockdown, error code %d.\n", ldret);
    goto leave;
}
lockdownd_start_service(lckd, "com.apple.instruments.remoteserver", &service);

if (!service || service->port == 0) {
    printf("ERROR: Could not start instruments.remoteserver service!\n");
    goto leave;
}
//Start the instruments service
if (instruments_client_new(device,service, &instClient) != INSTRUMENTS_E_SUCCESS) {
    printf("ERROR: Could not connect to instrumets \n");
    goto leave;
}

//Read capabilities message
char* buf = (char*)malloc(CAP_SIZE);
uint32_t bytes_read;
instruments_receive(instClient, buf, 32 ,&bytes_read);
free(buf);

//Send command to send cap message (again)
instruments_command_send(instClient,capMessage, CAP_MSG_SIZE, &sent);
num_bytes = CAP_MSG_HEADER;
buf = (char*)calloc(num_bytes, 1);

//Receive the header - we still bypass SSL
instruments_receive(instClient, buf, num_bytes,&bytes_read);
free(buf);
 //The above always fails with SSL_SHUTDOWN.

And this is the part for creating a new client, just like some of the other services.

LIBIMOBILEDEVICE_API instruments_error_t instruments_client_new(idevice_t device, 
lockdownd_service_descriptor_t service, instruments_client_t *client)
{
    service_client_t serv_client = NULL;
    instruments_error_t err = instruments_error(service_client_new(device, service, &serv_client));
    if (err != INSTRUMENTS_E_SUCCESS) {
        return err;
    }

    instruments_client_t client_loc = (instruments_client_t) malloc(sizeof(struct 
    instruments_client_private));
   client_loc->parent = serv_client;
   client_loc->parent->connection->bypass = 1;
    *client = client_loc;

   return INSTRUMENTS_E_SUCCESS;
}

@JonGabilondoAngulo anything you can see that I am doing incorrectly?

Btw the bypas code is in here (idevice.c)

LIBIMOBILEDEVICE_API idevice_error_t idevice_connection_receive_timeout(idevice_connection_t 
connection, char *data, uint32_t len, uint32_t *recv_bytes, unsigned int timeout)
{
    if (!connection || (connection->ssl_data && !connection->ssl_data->session) || len == 0) {
        return IDEVICE_E_INVALID_ARG;
}
//Bypass here - Set when creating the client.
if (connection->ssl_data && (!(connection->bypass == 1)) ) {

@JonGabilondoAngulo Never mind fixed it - Silly mistake as usual. Realised that I had bypassed only receive and not send. Now added bypass to send as well and it works now. SO previously it was sending using SSL write and that might be the reason for the SSL_SHUTDOWN reply.
Thanks for your help.

@nikias @JonGabilondoAngulo @karthikgbbristol
Hi guys, any updates?
Tried to implement something like @karthikgbbristol described above but no luck for now.
Going to go deeper though

@maznikoff What issues are you facing - As I mentioned about, as long as you ensure that both send and receive use usbmuxd_send/usbmuxd_receive, it should work. I have had iOS 13 working the last couple of weeks.

@karthikgbbristol may be I don't understand something or did something incorrectly but in my case I could not start application using idevicedebug. I tried the code written above but it changed nothing.
debugserver_client_send_command uses idevice_connection_send, but adding some checks to bypass changed nothing because connection->ssl_data is NULL so idevice_connection_send chains to internal_connection_send (the same for receive).

Also compared responses which sends me debugserver on iOS 12 and 13.
iOS 13:
idevicedebug run com.test.test
sending encoded command: $QSetMaxPacketSize:31303234#63
received char: a
received response: (null)
sending encoded command: $QSetWorkingDir:2F707269766174652F7661722F6D6F62696C652F436F6E7461696E6572732F446174612F4170706C69636174696F6E2F32324636303743452D413745422D344139422D384330342D363137393839424332314635#57
received char: 7
received response: (null)
sending encoded command: $A194,0,2F707269766174652F7661722F636F6E7461696E6572732F42756E646C652F4170706C69636174696F6E2F32414541413646322D343342422D344141302D424536432D4638393433444341314641452F736C656570792E6170702F736C65657079#81
received char: $
sending encoded command: $qLaunchSuccess#A5
received char: $
received response: E02
ERROR: 02

iOS 12:
idevicedebug run com.test.test
sending encoded command: $QSetMaxPacketSize:31303234#63
received char: +
received response: OK
sending encoded command: $QSetWorkingDir:2F707269766174652F7661722F6D6F62696C652F436F6E7461696E6572732F446174612F4170706C69636174696F6E2F30423042413034302D434634352D343444382D384634412D333341364645354139313638#47
received char: +
received response: OK
sending encoded command: $A194,0,2F707269766174652F7661722F636F6E7461696E6572732F42756E646C652F4170706C69636174696F6E2F43453143424439302D314630302D343439422D393631452D3638384442313241363532382F736C656570792E6170702F736C65657079#8C
received char: +
sending encoded command: $qLaunchSuccess#A5
received char: +
received response: OK
sending encoded command: $Hc0#DB
received char: +
received response: OK
sending encoded command: $c#63
received char: +
received response: O4e65772074696d652069732031320d0a

What is the 'a' and '7' that I receive on iOS 13? What's error 02?

@nikias asked "Also which other services except debugserver don't work (after the most recent commits)? What services don't use SSL? From what I can tell all are forced to use SSL now."
The ones we know needs the ssl bypass is com.apple.testmanagerd and com.apple.instruments.remoteserver

hi,
@nikias the issue here is that SSL shutdown should be not used in case service is switching to plain text mode after handshake. as SSL shutdown api will produce SLL traffic(shutdown message) that will be received by plain text service and considered as protocol data (gdb one).
submitted #860, currently we use it as patch in mobivm/robovm project and it seems to be working.

@dkimitsa Thanks! I just tested your patch in the windows build, and it works indeed!

Hi,
Just a user's request to add this dkimitsa's patch to master. It would make my life easier :)

I can confirm that even after applying the bypass_ssl patch #860, the Error 02 issue exists for iOS 13.2 devices. Also reported at #872

Sorry, I can confirm that, for me, it fixes the issue.

I have updated to the latest code from libimobiledevice and libusbmuxd and still get the SSL SHUTDOWN.

Following is the sequence of calls I make (I have written a stub program that just reads the capability message using the bypass, then sends a simple command (another request for capabilities) and reads the response. When reading the response I always get a SSL SHUTDOWN message. Fails every time on my iPhone 6s Plus.

//Create new idevice
if (IDEVICE_E_SUCCESS != idevice_new(&device, udid)) {
    printf("No device found, is it plugged in?\n");
    return -1;
}
//Create lockdown client with handshake
 if (LOCKDOWN_E_SUCCESS != (ldret = lockdownd_client_new_with_handshake(device, &lckd, 
 "ideviceinstruments"))) {
    printf("ERROR: Could not connect to lockdown, error code %d.\n", ldret);
    goto leave;
}
lockdownd_start_service(lckd, "com.apple.instruments.remoteserver", &service);

if (!service || service->port == 0) {
    printf("ERROR: Could not start instruments.remoteserver service!\n");
    goto leave;
}
//Start the instruments service
if (instruments_client_new(device,service, &instClient) != INSTRUMENTS_E_SUCCESS) {
    printf("ERROR: Could not connect to instrumets \n");
    goto leave;
}

//Read capabilities message
char* buf = (char*)malloc(CAP_SIZE);
uint32_t bytes_read;
instruments_receive(instClient, buf, 32 ,&bytes_read);
free(buf);

//Send command to send cap message (again)
instruments_command_send(instClient,capMessage, CAP_MSG_SIZE, &sent);
num_bytes = CAP_MSG_HEADER;
buf = (char*)calloc(num_bytes, 1);

//Receive the header - we still bypass SSL
instruments_receive(instClient, buf, num_bytes,&bytes_read);
free(buf);
 //The above always fails with SSL_SHUTDOWN.

And this is the part for creating a new client, just like some of the other services.

LIBIMOBILEDEVICE_API instruments_error_t instruments_client_new(idevice_t device, 
lockdownd_service_descriptor_t service, instruments_client_t *client)
{
    service_client_t serv_client = NULL;
    instruments_error_t err = instruments_error(service_client_new(device, service, &serv_client));
    if (err != INSTRUMENTS_E_SUCCESS) {
        return err;
    }

    instruments_client_t client_loc = (instruments_client_t) malloc(sizeof(struct 
    instruments_client_private));
   client_loc->parent = serv_client;
   client_loc->parent->connection->bypass = 1;
    *client = client_loc;

   return INSTRUMENTS_E_SUCCESS;
}

@JonGabilondoAngulo anything you can see that I am doing incorrectly?

Btw the bypas code is in here (idevice.c)

LIBIMOBILEDEVICE_API idevice_error_t idevice_connection_receive_timeout(idevice_connection_t 
connection, char *data, uint32_t len, uint32_t *recv_bytes, unsigned int timeout)
{
    if (!connection || (connection->ssl_data && !connection->ssl_data->session) || len == 0) {
        return IDEVICE_E_INVALID_ARG;
}
//Bypass here - Set when creating the client.
if (connection->ssl_data && (!(connection->bypass == 1)) ) {

Could you please show how you realize the function "instruments_receive" ? @karthikgbbristol

@anshumanchatterji using an Ubuntu 18.04.3 LTS I'm not able to launch an app in debug mode in iOS 13.3. What is your setup and iOS version ?

@hduarte The libimobiledevice version included in Ubuntu 18.04 is pretty old (AFAIR from 2017). At the moment you need the latest version from master from iOS 13.

Be aware that simply cloning libimobiledevice and compiling it will not work because also a lot of dependencies are too old to compile the latest version of libimobiledevice.

Additionally updating all dependencies and compiling+using the latest master branch may not be sufficient as there is a bug in this version (see #872 for details) that affected idevicedebug.
There is a PR that fixes this issue (liked in the issue 872) but for an unknown reason the maintainer of libimobiledevice seem to ignore this PR.

if i remember correctly, to launch app in debug mode (with debugger attached), you need first mount developer support image for ios.
apple has not released developer support image for ios 13.3, so currently it's not possible to do live debug on ios 13.3 device from xcode. (i've seen some hackish method, where people tried to put 13.2 into 13.3 iphone, but never got it working).

@mexmer mounting 13.2 does work, actually Xcode does same.

@mexmer The 13.2 images work fine in my environment. Xcode also seem to use them as debugging via Xcode works with 13.3 devices and there are no other developer images accessible to Xcode.

for me it throws error when i try to mount 13.2 image on my 13.3 ipad air 2

xcode beta now has images for 13.3 and 13.4

@JonGabilondoAngulo any luck starting the com.apple.instruments.remoteserver service on iOS14 devices. I am getting an INVALID_SERVICE error fwhen trying to start the lockdown service.

Just wondering if there's any chance that @JonGabilondoAngulo, @karthikgbbristol or others would be able to share their implementation of communication with com.apple.instruments.remoteserver anywhere. I'm sure it would be greatly appreciated by many people.