ADB-USB Wombat Manufacturing Trouble
Today I received a new batch of Wombat boards from my manufacturing partner. Too bad none of them work! Connected devices aren’t recognized, in either ADB or USB input modes. Oops.
After some lengthy investigation, I discovered that the main crystal oscillator speed on these new boards is about 3.69 MHz, but it should be 8.0 MHz. At 3.69 MHz the board will not work. But the oscillator seems to be an 8 MHz one (at least it has “8.0” in the part number stamped on it), and the associated capacitors and resistors measure roughly the same in-circuit values as a known-good board, so it’s unclear why I’m getting 3.69 MHz oscillation.
Problems happen, but this particular problem should have been caught by the testing procedure that I designed, before the boards were ever shipped to me. I re-ran the test procedure here, using a copy of the test harness that I previously gave to the manufacturer, and sure enough all five of the boards I tried failed the test. So either there’s some discrepancy between how I’m testing and how they’re testing, or they did the tests incorrectly, or didn’t run the tests at all.
The hardware and the test process are identical to the previous three Wombat batches made by the same manufacturer, but the person overseeing the project changed this time.
This is my nightmare scenario, and I’m not sure where to go from here. With some more bench time, I could probably determine exactly why the oscillation speed is 3.69 MHz. Perhaps these boards could be fixed with some hot air rework and component replacement, but who’s going to do that? I’m not about to desolder and replace crystal oscillators on hundreds of boards by hand, to fix someone else’s mistake. Returning the whole batch for rework in China would be prohibitively expensive in shipping and tariff costs. Even if the manufacturer agrees to stump for a complete re-do of the manufacturing, I’ll still be out roughly $750 for the cost of ICs I purchased and provided to them.
Read 22 comments and join the conversation22 Comments so far
Leave a reply. For customer support issues, please use the Customer Support link instead of writing comments.
That’s bad news. Sounds to me likely that your assembly house got some duff oscillators from their suppliers. As 3.6864 MHz is a standard frequency for serial communications, perhaps mismarked parts.
Hopefully the manufacturer will step-up and take care of the additional costs for a redo production (including the IC’s + misc. parts) run. At the very least they should cover what they charge you plus shipping. Should they do the latter, and the fix is easy enough, sell them at a discount to those willing to do the soldering work themselves?
Yup, it looks like Ed is right. I used a hot air tool to remove the crystal from one of the boards, and replaced it with another 8.0 MHz crystal, and now the board works fine. The part number of the bad crystal is ECSR8.0DUX.
I’m annoyed that the manufacturer used bad/wrong crystals, but I know mistakes happen. I’m more annoyed that the testing procedure apparently wasn’t done correctly, or done at all, because it would have caught this problem. So now I’m waiting to hear what kind of remedy or resolution the manufacturer will agree to.
Bad board with questionable crystal: https://www.bigmessowires.com/wp-content/uploads/2019/08/DS1Z_QucikPrint1.png
Good board from previous batch (same manufacturer): https://www.bigmessowires.com/wp-content/uploads/2019/08/DS1Z_QucikPrint2.png
I’d buy one as a “scratch’n’dent” with no warranty and “you get the board and a crystal, good luck”. If the manufacturing house won’t fix it, that would maybe be a viable solution to help you recoup costs.
I agree, I’d be more than willing to buy a scratch and dent board!
I’d buy a few as well and swap the crystals myself.
Does wombat already support the apple keyboard ?
I had a similar issue with a bunch of boards I had manufactured in which they soldered a 10K ohm resister pack in with the common ground in the wrong pin. Luckily they didn’t realize the pack needed to go in one way an the “law of averages” was on my side for once and out of 200 boards, I only had to rework about 40 of them. I contacted the manufacturer and they apologized, but it was clear they wouldn’t take care of the fix. My daughter learned how to de-solder and re-solder that weekend. I figured I could at least pay her to fix them for me. I gave them complete instructions including how to install the RPs in big bold yellow highlighted red letters. This really sucks.
Yep, would also buy a dodgy one at a discount and fix myself.
If the manufacturer does not reslove, then put them up for sale please!
I have a reqork station & hot air reqork station for surface mount work.
Let me know if I can be of assistance. -Bob
Ahm, make that rework, not reqork. :-\
Sorry but I can’t really afford to resell these as scratch-n-dent discount boards. The crystals will all be replaced and the boards retested before sale. The only question now is who will do it and who will pay for it.
The manufacturer asked me to ship back a small number of boards, for them to confirm the problem. Then they asked me to get a quote from a local electronics shop for crystal-swapping the rest of them, and they’ll pay for it if the quote isn’t too high. Labor costs are very high in the Silicon Valley area where I live, so that might be a challenge. I have a hot air rework tool so I could potentially do it myself, but it’s slow and tedious work and not something I enjoy – that’s why I paid somebody else to do the assembly.
Jay, thanks for your story. My daughter also has some soldering experience and could potentially help with this, although she’s a busy high school student. Bob Brown, if you might be seriously interested in doing the work, please send me an email and we can discuss the details.
I think Ed S guessed correctly – the frequency of the bad crystals measures 3.686 MHz, to the resolution limit of my frequency-counting multimeter. Just to be 100% sure these aren’t 8 MHz crystals that are somehow finding stable oscillation at 3.686 MHz due to some weird defect on the Wombat board, I’m going to take one of the desoldered crystals and use it with a breadboard ATMEGA or in a stand-alone oscillator circuit, and see what frequency it oscillates. If anyone knows a simpler way of finding the resonant frequency of a crystal in the single-digit MHz range, please let me know.
Steve, You can always take a good unit and measure the frequency & see if it tops off to the same frequency as a bad one… Might save you a little time in determining if the freq. counter is steering you in a wrong direction.
Update: the manufacturer found some extra crystals from this project in their warehouse and tested again, and confirmed they received a batch of mislabeled 8.0 MHz crystals that are actually 3.69 MHz. The explanation why testing didn’t catch the problem was blamed on a new test operator, which honestly sounds fishy to me, but they apologized profusely and agreed to ship back and rework the boards and re-test at their site and at their expense.
That’s a result! It should fix that testing ‘gap’ too, which you mightn’t have found out about.
I purchased one of these cheap tester kits from banggood. It has been great as I have some surplus parts with some that have worn labels, etc. It correctly identified all of the frequencies of the parts that are labeled. I buy these types of kits just to build them with my son, but some are great tools to have on the bench when completed. 🙂 https://banggood.app.link/7WgCt6bAcZ
Steve, I’d be happy to do this rework for you for a very reasonable price. I can rework ~50 boards a weekend, and live in Sunnyvale. Please reach out to me directly. I live in the north bay and have the required tooling to do this quickly.
Thanks, however the boards are already back with the manufacturer. I’m looking at early to mid-September to have them reworked and shipped back.
Interesting numbers: before returning them to the manufacturer, I did get three local quotes for the rework job from professional electronics houses. Prices ranged from $5 to $15 per board for the rework and retest.
Glad you were able to get them fixed – I will purchase at least one after they’re back 🙂 Keep us posted thanks!
It’s back in stock now.
I’ve started to switch over to sitime MEMS OSCs on my boards… slightly less headache with them IMO. And nice DFN footprints which are easier to solder to reliably than some of the crystals out there.