Cannot Boot to the IPXE After Restart/Shutdown the System on Comet Lake H about ipxe HOT 16 OPEN

Again: this issue is probably stuck unless someone is able to provide remote access to an affected system (or ship an affected system to me).

On the subject of remote access: I have a project in progress at https://github.com/ipxe/keyflogger to make it possible to use any Linux machine as a remote IP-KVM, which should help with remote debugging in future. The initial run of 10 prototype PCBs is currently being assembled.

from ipxe.

@ctony1 Based on prior experience with the i219, this is the kind of problem that would likely take me a couple of days of debugging with direct access to hardware that reproduces the problem. It's not something I can viably remotely guide you through fixing yourself, sorry.

from ipxe.

@daberkow can you suggest a cheap (sub-$100) machine I can source that would reproduce this problem? Alternatively, are you able to offer IP-KVM access for a few days to a suitable machine you have available?

from ipxe.

@mcb30 Sadly I had to return the laptop to someone else to do some work on. It seems this chipset is part of the Z490 series of intel motherboards, and some 10th gen Intel Dell systems like Dell OptiPlex 7080 according to intels forum. I was running it on a Precision 7550. I can not find a PC let alone just a motherboard under $300 right now with this chipset.

I think we maybe stuck for now unless later I get one and can get you access or something comes up with its chipset. Thanks for the help, it seems to be an Intel 10th/11th gen ethernet chipset that is used more starting around August 2020. Someone told me it was a 2.5gbps interface, but I can not find any information to back that and Dells website seems to say its 1gbps, when it was here I had it on a 1gbps switch.

I currently have a working solution with retrying DHCP with an embedded script until it returns a bootfile. Appreciate all the work you do here.

from ipxe.

What link speed does the switch support and detect?

from ipxe.

Hi NiKiZe
Can you kindly help tell how to check it. We tried different switch or connect the server directly with LAN cable, all failed.
We have traces as attached. Can you help review them? Thanks.

from ipxe.

@ctony1 There is no i219v-10 PCI ID recorded in the iPXE codebase. Please share the modifications that you have made to iPXE, since we cannot help you debug unknown code.

from ipxe.

@ctony1 There is no i219v-10 PCI ID recorded in the iPXE codebase. Please share the modifications that you have made to iPXE, since we cannot help you debug unknown code.

HI Michael
Sorry I closed the case by mistake.
I added the PCI ID manually to intel.c.
PCI_ROM ( 0x8086, 0x15e3, "i219lm-5", "I219-LM (5)", INTEL_I219 ),
PCI_ROM ( 0x8086, 0x0d4c, "i219lm-10", "I219-LM (10)", INTEL_I219 ),
PCI_ROM ( 0x8086, 0x0d4d, "i219v-10", "I219-V (10)", INTEL_I219 ),
PCI_ROM ( 0x8086, 0x1f41, "i354", "I354", INTEL_NO_ASDE ),
PCI_ROM ( 0x8086, 0x294c, "82566dc-2", "82566DC-2", 0 ),
PCI_ROM ( 0x8086, 0x2e6e, "cemedia", "CE Media Processor", 0 ),
};

Also change the general.h to support PXE image.
#define IMAGE_NBI /* NBI image support /
#define IMAGE_ELF / ELF image support /
//#define IMAGE_MULTIBOOT / MultiBoot image support /
#define IMAGE_PXE / PXE image support /
#define IMAGE_SCRIPT / iPXE script image support /
#define IMAGE_BZIMAGE / Linux bzImage image support /
//#define IMAGE_COMBOOT / SYSLINUX COMBOOT image support /
//#define IMAGE_EFI / EFI image support /
//#define IMAGE_SDI / SDI image support /
//#define IMAGE_PNM / PNM image support /
//#define IMAGE_PNG / PNG image support /
#define IMAGE_DER / DER image support /
#define IMAGE_PEM / PEM image support */

Please refer to the attachment about the code change.
code.zip

from ipxe.

@ctony1 Thank you for clarifying. The i219 has an extremely broken reset mechanism: see commit 546dd51de for details.

It is very possible that the i219v-10 has a subtle variation on this brokenness that would require further elaboration of the existing workaround.

from ipxe.

@mcb30 Hi Michael, thank you for sharing the information. We use the latest code. The native code has already implemented the workaround you share.
/* Set undocumented bit in FEXTNVM11 to work around an errata
* in i219 devices that will otherwise cause a complete
* datapath hang at the next device reset.
*/
Is there anything else we can try?

from ipxe.

Hi,

I got a Dell Precision 7550, it wanted in on the fun. It has an Intel i219lm-11... PCI 0x8086, 0x0d4c... I was able to get this card to work by embedding a script. It would be nice to get this fixed, but I understand how hard this remote debugging can be. I have tried marking it as INTEL_I219, INTEL_NO_PHY_RST, if you have any other things I can try let me know. My C++ skills are not super fantastic, but I am happy to try things while I have a system here from work. I am running ProxyDHCP, and it seems to some of the time get the info, some of the time not.

#!ipxe

echo Opening Connection
ifopen net0
sleep 2

:retry_dhcp
echo Attempting DHCP
dhcp && isset ${filename} || goto retry_dhcp
echo Booting from ${filename}
boot ${filename}

from ipxe.

@daberkow From what you've described, your issue sounds more like a DHCP server configuration problem than a driver problem. To confirm, could you share a screenshot of a boot (using your embedded script) that shows Attempting DHCP appearing more than once?

from ipxe.

@mcb30 It looks like that, but I have run iPXE on this network without issue for 7 years or so. I have an old Precision M6800 that boots fine with BIOS or EFI boot files. I have booted hundreds of servers on this network without issue on the stock image. I am getting some of https://ipxe.org/err/3c654083 and https://ipxe.org/err/380f60 . (Sidenote thanks for all you do here)

from ipxe.

@mcb30 It looks like that, but I have run iPXE on this network without issue for 7 years or so. I have an old Precision M6800 that boots fine with BIOS or EFI boot files. I have booted hundreds of servers on this network without issue on the stock image. I am getting some of https://ipxe.org/err/3c654083 and https://ipxe.org/err/380f60 . (Sidenote thanks for all you do here)

Thank you for providing the error URLs. The https://ipxe.org/err/380f60 is harmless (indicating just some broadcast UDP packets on your network), but the https://ipxe.org/err/3c654083 points directly to an unexpected interrupt happening in intel_poll() on line https://github.com/ipxe/ipxe/blob/5bdb75c9d/src/drivers/net/intel.c#L886

Could you build with DEBUG=intel and report the INTEL 0x.... unexpected ICR 0x.... debug message?

Thanks.

from ipxe.

INTEL 0x405166f0 unexpected ICR 00000106 I have photos of it working, and of it not working if those could help, I can upload them. It seems random when it is open or up enough to get a proxydhcp setting, vs when it doesnt. Some of the time I see INTEL 0x405166f0 unexpected ICR 00000102 instead as the link comes up and INTEL 0x405166f0 link status is 40080483

Right after no more network devices nothing to boot I see: INTEL 0x405166f0 link status is 40080480 then 2 INTEL 0x405166f0 unexpected ICR 00000100.

from ipxe.

I believe I also have a similar issue but the model is i219v-2. I tried this machine with undionly, snponly, ipxe.efi both in bios and UEFI, but all of them have the same problem. other machines work fine in the same network.

from ipxe.