Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qualification tests for revC2: pre INA233 #220

Closed
22 tasks done
electroniceel opened this issue Oct 24, 2020 · 34 comments
Closed
22 tasks done

Qualification tests for revC2: pre INA233 #220

electroniceel opened this issue Oct 24, 2020 · 34 comments
Assignees
Labels
hardware Component: hardware qualification Lifecycle stage: qualification revC Hardware revision: C

Comments

@electroniceel
Copy link
Member

electroniceel commented Oct 24, 2020

This issue is a task list of things to test for qualification of revC2.

This issue lists the tests that should be able to do without firmware support for the new INA233 ADC (pre issue #219).
There is issue #221 for tests that need support for the INA233 in firmware.

PCB/Mechanical

  • Use a addon board with soldered-in port plugs, does it fit in revC1 and revC2? (the spacing was changed a little)
  • Check all silkscreen markings, do they match the actual pins/ports (known issue: Silkscreen labels swapped on SCL and SDA testpoints #250)
  • Check new LVDS connector. Does the recommended mating connector still work well?
  • Check height of cap and reset button. Does the laser cut "case" still work or will it get in the way of addon boards?
  • Check how hard it is to replace/rework the USB-C socket. There is some flood fill in the layers that might make this harder than necessary.

USB/Power

  • USB communication works with USB-C cable plugged in both orientations
  • Powering via USB works from a USB-C host with a fancy cable with electronic marker
  • Measure current idle draw of Glasgow, compare to revC1, differences reasonable?
  • Run benchmark for some time and compare to revC1, there should be no significant difference

LEDs

  • LED brightness is similar among the LEDs, maybe with exception of the ERR led (@esden : assuming the prototypes use the final led part nos)

Reset

  • Measure rising voltage (+5V rail) the reset is released at, is it enough for the lowest voltage the usb spec allows?
  • Measure falling voltage (+5V rail) the reset is activated at, is it enough for the max. drop of the 3v3 LDO?
  • Measure time between press (!) of reset button and falling voltage of (enabled) Vio a and b. Is it reasonably fast for a emergency power off?
  • Measure time between rise of the internal 3v3 rail and release of reset for the FX2. Is it > 5 msec?
  • Short internal 3v3 rail, is the FX2 reset triggered?
  • Remove short on internal 3v3 rail, is the FX2 reset released >5 msec later?
  • Is the new reset circuit sensitive to EMI? Improvise some EMI, see if accidental resets are issued

EEPROMs

  • Storing the FX2 firmware works
  • Recover shorting works
  • Storing FPGA gateware works using glasgow flash <applet>

Short circuit behavior

Level Shifters

  • New shifter footprints: Vio to 5V, pin to output, short the pin. Keep shorted for a few hours. Does the shifter survive? Measure temp.
@electroniceel electroniceel added revC Hardware revision: C hardware Component: hardware labels Oct 24, 2020
@whitequark
Copy link
Member

  • TODO: do we already have support for it, I never tried it?

Yes we do, glasgow flash <applet> will do everything necessary.

@whitequark whitequark added the qualification Lifecycle stage: qualification label Nov 23, 2020
@electroniceel
Copy link
Member Author

@esden tested reworking the USB-C connector live on twitch.

Rework was not unreasonably difficult, just needed regular hot air.

@esden
Copy link
Member

esden commented Dec 7, 2020

I tested point one "Use a addon board with soldered-in port plugs, does it fit in revC1 and revC2? (the spacing was changed a little)" and the PROM reader addon fits perfectly on both versions of the board.

@electroniceel
Copy link
Member Author

Current consumption:

test revC1 mA revC2 mA
idle 34.8 36.4
firmware loaded 44.3 45.0
uart gw running 57.3 58.0
vga output 55.6 57.2

These differences are reasonable and can be explained for example by different led resistors and part variance.

@electroniceel electroniceel self-assigned this Dec 30, 2020
@electroniceel
Copy link
Member Author

Benchmarks:

revC1:
I: g.applet.internal.benchmark: running benchmark mode source for 953.674 MiB
I: g.applet.internal.benchmark: mode source: 41.39 MiB/s (331.09 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode sink for 953.674 MiB
I: g.applet.internal.benchmark: mode sink: 41.64 MiB/s (333.08 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode loopback for 953.674 MiB
I: g.applet.internal.benchmark: mode loopback: 41.62 MiB/s (332.95 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode latency for 953.674 MiB
I: g.applet.internal.benchmark: mode latency: mean: 740.00 µs stddev: 213.11 µs worst: 48991.92 µs

revC2:
I: g.applet.internal.benchmark: running benchmark mode source for 953.674 MiB
I: g.applet.internal.benchmark: mode source: 41.39 MiB/s (331.13 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode sink for 953.674 MiB
I: g.applet.internal.benchmark: mode sink: 41.64 MiB/s (333.08 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode loopback for 953.674 MiB
I: g.applet.internal.benchmark: mode loopback: 41.63 MiB/s (333.03 Mb/s)
I: g.applet.internal.benchmark: running benchmark mode latency for 953.674 MiB
I: g.applet.internal.benchmark: mode latency: mean: 744.14 µs stddev: 214.06 µs worst: 49607.51 µs

The differences are within the expected tolerances.

@whitequark
Copy link
Member

(Did we expect any changes in USB performance?)

@electroniceel
Copy link
Member Author

electroniceel commented Dec 30, 2020

No, we didn't expect any differences. If there were any, we'd have to investigate (bad layout, added capacitance and stuff like that could cause it).

This test was to make sure there isn't such stuff lurking anywhere.

@electroniceel
Copy link
Member Author

Reset thresholds of the new reset IC:
Rising (slow): 4.10 V
Falling (slow): 3.96 V
Measured at about 19 °C directly at the reset ic (behind the TPD3S014).

-> Matches the datasheet (which just specifies falling and not rising).

The TLV75533 has a max. dropout of 270 mV: ok
USB Vbus is allowed to go down to 4.40 V: ok

@electroniceel
Copy link
Member Author

Vio off time when pressing the reset button:

Worst case measurement: 5 V Vio, no load

Falling logarithmically,
5 V -> 1 V: 8 ms
5 V -> 300 mV: 16.2 ms

With 100 mA load:

Falling logarithmically,
5 V -> 1 V: 320 µs
5 V -> 300 mV: 610 µs

The time seems to be dominated by the voltage regulator switching off and the output capacitor to discharge.

Compared to the operator hitting the button this reasonably fast.

@electroniceel
Copy link
Member Author

Reset for the FX2 crosses the VIL level (800 mV) about 5.4 ms after the 3v3 rail reaches 3.0 V. This satisfies the requirements of the datasheet.

This is the same for initial powerup and after a short on 3v3.

A short on 3v3 instantly pulls low the reset.

@electroniceel
Copy link
Member Author

Height of cap and reset button:

The highest component on the prototype is the sync connector. But it was manually rised because of a component mismatch at @esden. The proper part is lower, Next lower part is the polymer cap (about 1mm).

But even with the high sync connector a 2.5 mm thick acrylic laser cut case works fine (top of acrylic = top of port shroud).

@electroniceel
Copy link
Member Author

Level shifter shorted from 5V.

Temp. increase over ambient by 19 K. The 33 R series resistor even gets warmer than the shifter: 31 K. Current flowing 102 mA,

After 12 hours the temp increase was still the same, also still 102 mA flowing.

The shifter still works fine for input and output, edges look similar on the scope. No measurable current flowing into the shifter in input mode. No increased supply current measurable.

@marcan
Copy link
Member

marcan commented Dec 31, 2020

Worth noting that IIRC I torture tested the old shifters without the protection resistors for a day or two and they were still fine :)

@electroniceel
Copy link
Member Author

All the older revs already have the resistors. So did you bridge them for testing?

@marcan
Copy link
Member

marcan commented Dec 31, 2020

IIRC I soldered directly to the shifter pin.

@electroniceel
Copy link
Member Author

Ok, so it seems there is enough margin. The new shifter footprint has slightly worse thermal resistance than the one in revC1, but that doesn't matter because it is usually used with the resistors like I tested them.

@electroniceel
Copy link
Member Author

Redid the tests as in #135 with revC2:

Just shorting one port from 5V with a function-gen controlled fet is enough for triggering the new reset ic.

DS4_QuickPrint10
DS4_QuickPrint11

Yellow: internal 5V rail, measured at the polymer bulk cap
Magenta: Vio pin on port A

The results look even cleaner than what we had on revC1, no trace of oscillation. The reset ic triggers, Vio switches off and that's it.

I also measured a short 30µs load pulse:

DS4_QuickPrint12

The pulse length is still enough to drop the 5V rail to a reset condition. But the regulator is switched off cleanly and then the bleeding resistor begins to discharge Vio. So no trace left of the buggy still-conducting behavior from revC0.

@marcan
Copy link
Member

marcan commented Jan 1, 2021

Is this with or without the series resistor? Ideally we don't necessarily want glasgow to reset on a momentary Vio short, I would say. That's how revC1 worked. The regulators should current limit on that condition without killing the whole board.

@whitequark
Copy link
Member

whitequark commented Jan 1, 2021

Ideally we don't necessarily want glasgow to reset on a momentary Vio short, I would say.

I've been thinking about that, and at first, that was also my view. But for revC2 we went even harder for safety than we did before, adding a E-stop button (which resets the board) that you are expected to use any time you rewire anything. I think that momentary shorts triggering the E-stop behavior "automatically" are, at least, in line with that design direction. A short on a power rail is unquestionably a failure of some kind (it's certainly not a normal condition!), and failing safely is a nice property.

There is the separate question of whether INA233 can detect and abort momentary shorts before they can affect the board power rails. If yes, that should be our only supported mechanism for keeping the board alive across a momentary short.

@electroniceel
Copy link
Member Author

This is with the default series resistor and shunt (0.33 + 0.15 Ohms).

Not resetting the board on a short could be done by

  1. making the reset ic less sensitive
  2. increase bulk capacitance
  3. use another voltage regulator that limits faster / stricter
  4. increase series resistance to voltage regulator

I don't think any of these are particularly good options and safely switching off is the better option.

The INA233 isn't fast enough to prevent the 5V rail to drop. It works by constantly sampling and converting (delta sigma adc) Vsense and Isense (constantly switching between the two). If one measured value goes beyond the programmed limit, it can trigger the alarm. The shortest sampling interval is 140 µs, so 280 µs between two current measurements. A 30 µs short was enough for reset in the measurement shown above. So it is about one order of magnitude too slow.

@whitequark
Copy link
Member

Ack. I'm perfectly fine with safely switching off on a momentary short.

@marcan
Copy link
Member

marcan commented Jan 1, 2021

I'm concerned about large bulk capacitance connected across Vio spuriously triggering this behavior. Can we qualify how it behaves in this scenario? Especially during soft power up scenarios.

30μs quite honestly sounds to me like spurious glitch territory to me. I'd be fine with a shutdown after a 10ms short or such, but...

The point of the Vio current limiting behavior of the regulators was to insulate Glasgow from not just actual fault conditions, but also "general badness". Glasgow is obviously not a tool only meant to be used with well-behaved circuits, and I hotplug things all the time. If I have Glasgow in powering logic analyzer mode I don't want it to just die on me while I'm debugging something just because a new subcircuit got powered up and glitched Vio for a few microseconds.

If enabling Vio into 2000μF of capacitance results in a reset, I would consider that a problem. If externally connecting such capacitance while live does, that is still not great, but would not be as critical a flaw.

@whitequark
Copy link
Member

If enabling Vio into 2000μF of capacitance results in a reset, I would consider that a problem

Enabling into a short worked fine in my limited testing, though I'll defer to @electroniceel to check this more thoroughly.

@electroniceel
Copy link
Member Author

electroniceel commented Jan 6, 2021

Improvised EMI test:

  • Glasgow constantly running loopback benchmark. Throughput and any usb-related kernel messages are monitored
  • Glasgow connected to a stationary PC, IEC protection class I (GND connected to PE in the PSU)

A massive industrial relay (rated 4x 40 A, 24 V coils, coil current 0.5 A) is wired in oscillation mode using it's NC contacts.

Just the oscillating relay next to glasgow

No measurable effect

Connecting relay coil GND to Glasgow GND (PSU powering the coil isolated)

No measurable effect

Connecting relay coil GND to Glasgow GND (PSU powering the coil connected to PE)

In this setup a part of the coil current flows through the USB cable.

Sometimes leads to USB disconnects. Also the monitor sometimes flickers black and once the PC crashed. No reset of the FX2 was visible on Glasgow LEDs.

So this level of EMI is more than common PCs are immune against, so not a concern for Glasgow. But even then no spurious resets on Glasgow.

Coil of fluorecent lamp ballast switched with relay

Oscillating setup as above.
Coil wired in series with a incadecent bulb for current limiting, connected to mains.
Next to Glasgow
No measurable effect

GSM900 call directly next to Glasgow

Test was done in the basement, to provoke the (ancient) cell phone using max power.
In a cheap active speaker nearby I could loudly hear the GSM noise.
No measurable effect on Glasgow.

@whitequark
Copy link
Member

Sometimes leads to USB disconnects.

I'm fairly sure this happens because USB isn't fully differential and the DC level matters for things like SE0 detection.

Test was done in the basement, to provoke the (ancient) cell phone using max power.

Another good test I have found in the past is using a CB or commercial radio. 5 W seems to reset USB devices quite reliably.

@electroniceel
Copy link
Member Author

electroniceel commented Jan 6, 2021

Sometimes leads to USB disconnects.

I'm fairly sure this happens because USB isn't fully differential and the DC level matters for things like SE0 detection.

I'm fairly sure this happens because this was a massive level of EMI. Otherwise neither the monitor would flicker nor the whole PC would crash.

I use this machine on my electronics bench on a daily basis and it is a stable and well working machine.

Another good test I have found in the past is using a CB or commercial radio. 5 W seems to reset USB devices quite reliably.

I can try that too.

@electroniceel
Copy link
Member Author

Keying up a Baofeng at 145 MHz FM with (specified, not verified) 5 W directly above Glasgow

USB disconnect
At some antenna positions: Glasgow reset

Keying up a Baofeng at 145 MHz FM with (specified, not verified) 1 W directly above Glasgow

No measurable effect

Keying up a Baofeng at 434 MHz FM with (specified, not verified) 5 W directly above Glasgow

No measurable effect

@whitequark
Copy link
Member

Otherwise neither the monitor would flicker nor the whole PC would crash.

Oh yeah if the whole thing crashes you're right. I misread, sorry.

Keying up a Baofeng [...]

This tracks with my experiments.

@electroniceel
Copy link
Member Author

Keying up a Baofeng [...]

This tracks with my experiments.

And I don't see it as concern for Glasgow design, especially because going down to 1 W eliminates the problem.

@electroniceel
Copy link
Member Author

About the LEDs:

@esden told me that the LEDs he used on the prototypes are different parts than the ones specified in the BOM. He plans to use the same LED / resistor combinations as in the prototypes for the production run.

In my experiments I found the LEDs on the prototype to work well and evenly lit. But I don't have a prototype of the aluminium case so I couldn't test how they work for that scenario.

@whitequark
Copy link
Member

He plans to use the same LED / resistor combinations as in the prototypes for the production run.

Then it will be necessary to recalibrate the resistors. cc @marcan

@whitequark whitequark reopened this Jan 6, 2021
@electroniceel
Copy link
Member Author

As I wrote, the brightness levels of the LEDs on the prototype seemed fine to my eyes, so @esden running production with these resistors+LEDs would work for me.

The bigger issue I'd say is finding a brightness setup that works well with a bare board and also through the lightpipes on the al case. This is complicated by the fact that to my knowledge only @timonsku has a prototype of the case, and that prototype has a different height than the final cases will have (the prototype has the capacitor sticking out, the final one will not).

If further discussion is wanted, I'd prefer if we could create a separate issue for it.

@esden
Copy link
Member

esden commented Jan 6, 2021

I had a conversation with @whitequark about this today. Especially after yesterday's stream, I have to agree and change my position on this. We will have to at least adjust the resistors to increase the brightness. What I ended up mounting onto the prototypes is not good enough. I would rather not open the topic and delay things but unfortunately that is not possible. And yes we should probably open a dedicated issue to fixing the LED selection and associated resistors.

@electroniceel
Copy link
Member Author

@esden created issue #259 for further discussion of the LED issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hardware Component: hardware qualification Lifecycle stage: qualification revC Hardware revision: C
Projects
None yet
Development

No branches or pull requests

4 participants