Wow – and I thought the original phi mining article 8×7220 Phi System had gotten a lot of replies… man, was I wrong: the latest article on mining with the Asrock Rack 4×7210 system got a solid two thousand reads in the first 24 hours. Amazing.
Anyway – there were lots of questions on these systems (as well as on the Exxact system I mentioned earlier), and I’ll try to answer them all. I’ll try to answer every email I’m getting; but still, some questions seem to be recurring more often than others, and for those I think (hope?) it’ll be easier if I just put them onto the blog itself. I’ll start today with a two or three key ones, and will update this page (ie, not post a new one every time) as I get to it…. bear with me; I can only type so fast
Q: What is the wall power draw on those systems?
Probably the most asked question, and about an hour ago I decided it’s quicker to just drive to home depot and get a new Watt-meter than trying to reply to several emails (unfortunately I apparently fried my original one when I put it behind that 24kH monster machine I described earlier …. not a good idea).
Now with that brand new gadget, here’s what I got: My Asrock Rack 4×7210 system pulls pretty much 1300-1350W from the wall (on 110V, will update with 210 once I move it to my cohosting site). Of course that is under full load:
The 7210s have a TDP of 215W, so seems the rest of the system is pulling an additional 500Ws on top of that – way more than I would have expected, but still… 1.3KW for 11KH/s isn’t bad at all (that’s what my average GPU mining rig pulls as well, with significantly less output). The 7250s will likely pull more, but don’t have any asrack ones yet.
In my Exxact 4×7250 System I’m pulling more, apparently something around 1550W:
Note that is with the DIMMs and disks removed (see next post to come), but I doubt that’ll add more. Now why this pulls 200Ws more than the other one I don’t know – the TDP difference between 7210s (215W) and 7250s (250W) should only account for 140Ws, not 200… but then, I have them on 110Vs, which is where the PSUs won’t be at their utmost efficiency.
Q: Where can I get those systems?
Right now there’s two confirmed sources that I have used myself: one is Exxact Corp; the other is Asrock Rack. In particular the ~$3k system that the last post was about is from Asrock Rack.
For those that asked where to find the web page to order those from: Such a page does not exist (yet), but you can send an email to firstname.lastname@example.org . Note they “apparently” got quite some interest recently (wonder where this was coming from 🙂 ), so they may be short on stock.
In addition to Asrock and Exxact, you may also be able to get some such systems directly from Intel. I’ll post more when something on that front materializes more clearly, but at least if you want to buy more than just one or two systems I’m sure somebody would like to talk you (and I’d be happy to make the contact). Full disclosure: As my “About” page states I do actually work for Intel in my day job, but in a totally different capacity… (yes, “interesting” situation indeed, tell me about it).
Q: Why do some systems perform better than others?
As mentioned in an earlier post the default BIOS that these systems originally came with wasn’t optimal for the MCDRAM, but that can be fixed by using the right BIOS. Asrock Rack already found “the right” one; I’ll post it as soon as I receive it myself. Of course I also asked Exxact for one, but no reply yet (I only asked last night).
I sincerely hope Asrock will automatically put the right one onto future systems (in fact I’m 90+% sure they will), but jsut in case: you might want to tell them what the system is for, and ask if it has the right BIOS… just in case.
In addition, there also seems to be some OS influence as well – doesn’t make the slightest sense to me, but it seems Ubuntu can be up to 10% faster than CentOS – not sure if that’ll still be the case after the BIOS update, but at least for the old BIOS that difference has been confirmed by multiple users (as well as myself).
Q: Are those co-processors / GPUs / PCI cards ?
No. The x200 “Knights Landing” series (and now the x205 “Knights Mill” ones as well) are all “bootable” processors. You can get them in workstation form factor as well (e.g., from Colfax: http://dap.xeonphi.com/), but primarily they’re intended for High-Performance-Computing(HPC) / Supercomputing (e.g., the Stampede supercomputer has 6,400 of those Phis!), so their “natural habitat” is in rack-mount servers (e.g., in a 2U rackmount form factor). Either way, they are the “main CPUs” that go right onto the server motherboard. The big upside of this is that it’s so much easier to go “at scale” – find a co-location, pay them for their space and power, and just rack them up….
Note there are also a few isolated prototypes of PCI cards out in the wild (the so-called 7220s and 7240s), but this is a different discussion – and since they’re not freely available, anyway, please ignore them for now.
Q: What else do I need for this build ?
Well, that’s an almost trivial one: Nothing.
Though it’s significantly less fun than a “real” build (such as the one in my post with 7220 cards – that was fun!) these are ready-to-go rackmount systems. For the Exxact system the system comes with memory, disks, and OPA cards (none of which you actually need :-/); you can install linux on the disks, and run it, or take out the disks and use a pre-built bootable USB stick (will write more on that in a separate post).
The Asrock Rack system is a “barebone” system with just the chassis, PSU, and CPU – you don’t need either memory or disks, but you obviously do need something to boot off, so what I did is create myself a bootable USB stick with a linux distribution (and lukminer preinstalled in the boot script). So you do need those USB boot sticks (or to plug in your own disks – newegg has refurbished barracudas for $10 a pop!); but other than that it’s ready to go.
Be warned though they are not exactly quiet – they’re intended for data centers, not for “under the desk”, or “next to my TV”.
Q: Now why are the Asrock Systems so cheap?
Now that is an interesting question ;-). According to ark.intel.com the list price for 7210s is indeed around $1,900 per 7210; so one’d expect a four-node system to cost at least $8k just for the CPUs. However, there currently “seems” to be an active promotion going on where the “older” x200s (7210, 7250, etc) are availalable for a “lower” price – that may or may not be related to the fact that the newer x205 “Knights Mill” generation of the Phis is now out, but whatever it may be – right now they are quite steeply discounted. Asrock Rack (or Intel?) may or may not be able to tell you more about that… I can’t.
Note, of course, that if that is a special pricing promo it may or may not disappear at some point in time. Again, Asrock – or your favorite Intel distributor – may or may not know more.
Q: What miner are you using to get this performance?
Well – since this is the “lukMiner” blog you may not be surprised that I use the “lukMiner” miner :-). Pre-built binaries for the phis are available via this link ; and yes, I do take a developer share, so of course, I’m heavily biased towards these systems – I do not get any commissions from either Asrock, Exxact, Intel, or whoever (which is doubly weird because I work for Intel in my day job, in a totally different capacity!?) … but I do get a share if you guys are using lukMiner to mine on them. On the upside: That means I have a very high incentive to make it even better and even faster, and to also look into other coins….
Q: What coins is this for?
This can also be googled on the lukMiner google site, but just to mention it here as well: Currently lukMiner supports only cryptonight (monero, bytecoin, electroneum, sumo, …) and cryptonight light (aeon) coins.
As to other coins: Since the Phis are regular Intel CPUs you can, in theory, also run whatever other miner code you want to run… but be warned: The MCDRAM and the lots of cores will help, but unless a given code has been specifically optimized for the Phis you may not get the same profitability.
I myself am currently already working on a phi-optimized Ether miner, which of course would be a much “bigger” thing than just cryptonight (ether market cap is $85 billion, criptonight is 3.5 :-/)…. but that’s not ready yet. MCDRAM should help a lot – my current code makes 12ish MH/s and doesn’t even use a quarter of the MCDRAM bandwidth yet … but again, that’ll require some more work to look into. And of course, I’ll certainly post it if I get it running.
What performance should I expect for hardware XYZ?
I created a page on the google lukMiner site where I list – and continuously update – performance data for various platforms.
Can I also use the previous-generation 7120s, 5110s, etc?
Well, yes, lukMiner supports it …. but it’ll be significantly less profitable, to the degree that I’d advise against buying any of those. An overview of expected performance can be found on the respective lukMiner google site… do your math.
As mentioned in the beginning I’ll keep on adding more questions&answers to the bottom of this page, but at least for now I’ll have to go back to looking after my machines …
As such: Happy mining!
17 thoughts on ““Frequently asked questions” on the latest article…”
You are a legend
why did you remove my question? was there something controversial about what i asked? i asked whether RAM was required with the asrock rack barebones server. you stated that “the Exxact system the system comes with memory” and “The Asrock Rack system is a “barebone” system with just the chassis, PSU, and CPU – you don’t need either memory or disks”. Sorry but this doesn’t make sense to me. If it doesn’t come with memory, why don’t you need to add memory?
hey, jj – I didn’t _remove_ your question, I just didn’t get to accepting it yet. Sorry, but I also have a day job ….
As to the question itself: maybe it’s my not being a native speaker, but I still can’t find where the original statement was in any way ambigiuous … but let me try to say it in a different, more verbose form:
a) the phis do _not_ require and DRAM DIMMS at all, because they have 16GBs of on-chip MCDRAM
b) the asrack systems do not contain any DRAMs / DIMMs on the board when you order them….
c) … and because the phis do not need the drams (that those systems dont have) you don’t have to add any, either, so everything is good.
All I wanted to make clear is that when people hear “barebone” they don’t think that significant parts (such as RAM) are still missing, and have to be added separately – which isn’t the case, because you don’t need it ….
BTW: Same goes for the disks: The asrack system doesn’t have any (exxact does), but you can live without them if you instead use a $5-a-pop USB stick (rather than a $150-a-pop SSD).
you mentioned that ram isn’t required when buying the $3k rack from asrock, but you also said that it doesn’t come with ram. could you please clarify? thank you
i’m very sorry if i came across as rude before. i didn’t notice that my comments were awaiting moderation. i thought they were posted and removed because i checked from a different computer. i am really really sorry about that.
anyway, thanks you did answer my question. i do appreciate the response and the work you are doing.
Lol; no prob ;-). Did my answer make sense? I know this whole “on chip MCDRAM” is confusing for many – in fact, I myself was not sure originally whether the phis would work with only this what-i-would-have-considered cache …. and indeed, it’s a wonderfully weird sight when you see a motherboard booting up without any dimms in it (I have a open K1SPE based build, too) …. it definitely does look as if something was missing ;-).
(But it does work!)
In the reply from AsrockRack to my inquiry, they mentioned that they would be replacing the 1600W PSU with a 1200W one… As per your Kill A Watt measurement, I would understand that the 1200W would not be sufficient to sustain the load of the 4 CPUs… Any thought?
No prob; there’s two PSUs in that machine, and they work together. I had to learn that myself – had previously always thought they’d be “redundant” in the sense that one backs the other if one _fails_ – but apparently they actually do work together. I’ve never seen this before (all my systems I had before had redundancy in the sense I just mentioned (i.e., first thing I always did is pull out one PSU to sell later on on ebay ), but in the Asrock system the two do work together.
In fact, that’s already the case in the machine I got last friday: If I have one PSU plugged in I can only run two (or three?) nodes under full load; but when I plug in the second PSU as well I can run all four.
Can you share what you paid for the Asrock rack system with shipping and tariffs included?
As I explained in another answer they paid my customs fees because of some (minor) oversight of theirs when they first sent me the invoice – totally minor thing, but took a while to sort out, so they volunteered to pay tarriffs to make good on it (un-prompted, btw!). Either way, I don’t know what it _would_ have been – but doubt they’d have stood for it if it was a significant cost.
Shipping was – I think – about $200 per unit.
LikeLiked by 1 person
Good to know, I was concerned about this as well.
Hi lukMiner, Can you report back the 7210 power consumption on 2100 please?
I’m not exactly sure what you means with “on 2100” ?
210V sorry 🙂
Problem is ASRock Rack wants payment now for delivery in X(unkown) timeframe. These need to be on-hand/in-stock for this model to work.
Yes, delays are always a worry when considering crypto mining. I bought some Litecoin miners that took 3 months to arrive, and we all know what the prices did between October and January. 😦
Finally got confirmation that my ASRock server has been shipped last week. I should have it in a few days. Only took them 3 months. Grrrrr …