Many of the current issues may be related. I just ran the PIDcontroller with the motor stress test while connected to the firmware debugger. The crash is logged on the logcat as:

13:34:58.898 AndroidRuntime   E  FATAL EXCEPTION: pool-2-thread-1 (Ask Gemini)
                                 Process: jp.oist.abcvlib.pidbalancer, PID: 19726
                                 java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: SerialTimeoutException on send. The serial connection with the rp2040 is not working as expected and timed out
                                 	at jp.oist.abcvlib.util.ScheduledExecutorServiceWithException$1.run(ScheduledExecutorServiceWithException.java:68)
                                 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                 	at java.lang.Thread.run(Thread.java:1012)
                                 Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: SerialTimeoutException on send. The serial connection with the rp2040 is not working as expected and timed out
                                 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
                                 	at java.util.concurrent.FutureTask.get(FutureTask.java:191)
                                 	at jp.oist.abcvlib.util.ScheduledExecutorServiceWithException$1.run(ScheduledExecutorServiceWithException.java:64)
                                 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
                                 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
                                 	at java.lang.Thread.run(Thread.java:1012) 
                                 Caused by: java.lang.RuntimeException: SerialTimeoutException on send. The serial connection with the rp2040 is not working as expected and timed out
                                 	at jp.oist.abcvlib.util.SerialCommManager.sendPacket(SerialCommManager.java:189)
                                 	at jp.oist.abcvlib.util.SerialCommManager.access$400(SerialCommManager.java:19)
                                 	at jp.oist.abcvlib.util.SerialCommManager$1.run(SerialCommManager.java:55)
                                 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
                                 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
                                 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:308)
                                 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
                                 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
                                 	at java.lang.Thread.run(Thread.java:1012)

This represents the serialConnection timing out after 10 seconds with no response from the robot.

On the robot end, nothing crashes in the firmware, but if I stop execution after the crash on Android and inspect the backtrace, I see

#0  i2c_write_blocking_internal (i2c=0x0, addr=0 '\000', src=0x0, len=0, nostop=false, timeout_check=0x0, ts=0x0) at /opt/external/pico-sdk/src/rp2_common/hardware_i2c/i2c.c:198
#1  0x10012524 in i2c_write_blocking (i2c=0x20001590 <i2c0_inst>, addr=96 '`', src=0x20041f94 "", len=2, nostop=false) at /opt/external/pico-sdk/src/rp2_common/hardware_i2c/i2c.c:242
#2  0x1000101e in set_motor_control (motor=MOTOR_LEFT, control_value=197 '\305') at /project/src/drv8830.c:102
#3  0x10004a72 in set_motor_levels (state=0x200234bc <rp2040_state_>) at /project/src/robot.c:149
#4  0x10005792 in handle_packet (packet=0x200029ac <incoming_packet_from_android>) at /project/src/serial_comm_manager.c:132
#5  0x1000566a in get_block () at /project/src/serial_comm_manager.c:83

Which represents a blocking i2c call to set the motor speed.

In summary:

Android sends a call to setOutput on the firmware and times out after 10s from no response
The firmware is stuck in a blocking call to change the motor speed in an i2c call to the motor driver chip. It won't respond to serialConnection requests as it is stuck in this blocking call.

The reason it gets stuck in the blocking call is still a mystery to me, but I suspect it has something to do with an overcurrent protection mechanism on the chip.

I set a breakpoint in the firmware here and it breaks here often actually. The value of fault_values has been observed to be:

Decimal	Char (escaped)	Binary (8-bit)	Fault
4	'\004'	00000100	UVLO (Though I would expect the last bit to also be 1)
0	'\000'	00000000	None
15	'\017'	00001111	OTS,UVLO,OCP,Fault

Comparing against the datasheet, these represent:

Depending on the situation.

Moreover, the assert statement here and here were hit crashing the firmware intentionally, though I cannot read these rp2040_logs for the moment as the never get to Android. Working on reading them via UART as a workaround.

I commented out the above assert statements and found it asserts to false here then. Commenting out this and the backtrace quoted above is the fault again.

All of this seems to point to one thing; the drv8830 motor driver chip is the issue. The firmware is trying to handle the various errors, but the drv8830 is just not responding to anything. I suspect it is going into some shutdown state as explained in the datasheet.

Finally I found a completely unrelated hardware issue on board #2 that was also causing crashes. (Yuji had marked this robot with a piece of tape, I'm not sure if this was to indicate this or something about the Transmitter/Receiver roles). Regardless, something was rattling inside the robot and when I shook it back and forth the firmware debugger would throw errors and disconnect. Taking the pcb out I found the ratling was on the PCB itself not some lose screw or the ratling from the plastic body. I de-soldered and removed the battery holder thinking there was something underneath, but nothing. The rattling can be heard only on this board. I compared against another and no rattling. I suspect the usb-c plug has some issue from all the jamming the phones in, but I have no replacements to test against for now. I can order others, but will try to simply desolder it and confirm the rattling is gone or not.

Robot freezes without speed updated #51

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions