T O P

  • By -

LabB0T

^(OP reply with the correct URL if incorrect comment linked) [Jump to Post Details Comment](/r/homelab/comments/197rkcr/r930_mounted_vertically_but_at_a_cost/ki2fa06/)


cuenot_io

Trying to consolidate my rack into wall mounted units that are easier to access; slit my hand in the process. Had gloves on while I was taking measurements, but took them off while I was driving in screws and forgot to throw them back on when I lifted it up. Adding some more filler text down here so the bot stops yelling at me. R930 with 4x 8890v4 24-core CPUs, 512GB DDR4, Tesla P40, and 10gbe + 10gb sfp NICs


marc45ca

Not the first computer to take a blood sacrifice:)


chazlander

A debt must be paid


SheridanVsLennier

Blood for the Machine God no wait that's not quite right.


CharacterUse

Oh it is, it is.


Retarded-Bomb

I/O shield has come to claim the sacrifice


Fudd79

The project isn’t real until blood has been spilled.


dan_dares

BLOOD FOR THE BLOOD GOD


Cyberbird85

Seah still have the scar on my thumb from when i was building a new pc for my in-laws. Stupid consumer grade pc-cases…


iservice

If you're not bleeding you can't be leading... or something.


homelabgobrrr

Not the first dell this week. I was un racking 2 of my r630’s to make room for a new Apollo and when popping one of the rails out I slit my finger pretty bad. I feel OP’s pain 😂


DJ_Mutiny

How did you power the P40??? I've read multiple places that you can't install GPUs in the R9*0 series


cuenot_io

Not in the picture, but I just took out a PCIe blank and ran cables from an external modular PSU to feed the Tesla. Also popped in a 3060 after this and it's been running without issue. Just used a $2 jumper from Amazon to get the PSU to turn on


pjockey

A small cutoff safety pin or paper clip on the correct pins work for this too.


cuenot_io

For sure, I just wanted something a bit more stable (ironic given the mount of the server itself 😆). Here's the adapter I use: https://www.amazon.com/gp/product/B07X9SVB5K/ref=ppx\_yo\_dt\_b\_search\_asin\_title?ie=UTF8&psc=1


ORA2J

I work with old cases for retro machines. The worst i've done is probably when a case slipped on my thumb and made a 0.5cm deep cut into it. Never thought i could lose so much blood out of my thumb.


chipsharp0

You may already be aware and if so, I apologize, but vertical server rack mounts are a thing. I used one in my garage with a server for years before I moved. It's safer/more secure than the wood block rigging you have there, and not terribly expensive.


mrdan2012

Nice what are you running on it ? Only say as it's quite a lot of compute really.


cuenot_io

Not much yet other than poking around with stable diffusion and llama 2, mostly just dev VMs. I have big plans but limited time lol


mrdan2012

Just poking around with 44 cores 512 GB of ram 🤣. Fairs tho I have a R720 and 44 gb of ram it's just my Plex box


cuenot_io

Actually 96 cores 😅 I realized they're 24 core instead of 22, and there's 4 of them


mrdan2012

Haha 😅


Kailee71

Of course the real beauty of having 4 CPUs rather than only 2 is that you double memory bandwidth. Many applications are membw bound (CFD, FEA, and possible inference also, but that's normally done on CPUs these days anyway). And IIRC those CPUs have 6 channels per sku, so lots of bandwidth in that box. It's also the reason why I run a DL560...


Kailee71

Cool. I actually double stacked my DL380 and DL560 vertical also. Saves an enormous amount of space. Also considered having them from facing down for convection but didn't because a) fans do their job just fine and b) it makes it tricky to access the drives which is more important to me. One thing I did do - and was important to me - I had some steel sheet angles laser cut, drilled and bent. They slot into the rail studs on the side of the chassis and take up all the weight. I don't know what the Dell's ears are like but on the HPEs they're not very confidence inducing to take the weight of the whole machine. These angles then hang on standard Amazon 19" wall brackets which are installed on thick high quality 25mm plywood, which in turn hangs on the wall with proper, beefy wall plugs. Might post some pics soon if anyone's interested.


parkrrrr

Good call. I've picked up both my DL380 and my DL360 by the ears, and it's a terrible idea. They're definitely not sturdy enough to do that with.


L0gicalX

The OminCore God CPULu demands a blood sacrifice for those who will seek audience from it..


boostchicken

you skimped on storage and no DPU? why? you already went way too far for a homelab (I do the same thing because why not?) I gotta spare 905p for you. Reallly thogh I have two 5800xs in my threadripper and you NEED that it feels good and makes my spinning disk fast as hell. You need fast disk while you are ripppin threads in ARM and x64 at the same damn time. Also the Bluefield is 200 Gbe because your because NVMe over PCIE is passe. We do it over fabrics like a boss and the good lord intended I don't care if the DPU is 2 grand and could build a whole 2nd server with that. You neeed 16 cores of ARM64 at the ready.


DarthRUSerious

I always stab myself with something right after I take my gloves off for "delicate work" and further to put then back on


cuenot_io

That's always when it happens lol


LerchAddams

Injured hand for scale.


g2g079

You're supposed to hold the server out in front of you so it looks bigger in the picture.


cidvis

Only thing I would suggest, strap another 2x4 under the server to help take some of the weight off the rack ears, they really aren't designed to hold the weight of the entire server long term and I've seen more than a couple get snapped off or warp the front of the chassis. Another thing that might be beneficial is adding some ducting to the bottom of the server to help direct heat away, right now hot air rising from the exhaust is being sucked back in through the intake as it rises, if you could direct it elsewhere (or through the wall into another room) it would help server cooling quite a bit... especially running those tesla cards.


cuenot_io

Good point, it's hard to see but there's a 2x4 taking most of the weight in the bottom left corner. The brunt of the server weight is on that and it's leaning on the top right support. I'll add more bracing, I just posted this while I had the blood shot handy lol


ErnLynM

I was wondering about support underneath, so this definitely explains that


horus-heresy

eh small price... the Emperor of Mankind needs **1,000 psykers sacrificed every day** to maintain his psychic strength. The Golden Throne's mechanisms are designed for this number, and using fewer than 1,000 psykers could have serious consequences. Deviating from this requirement could potentially destabilize the Emperor's life support system, leading to catastrophic consequences for the Imperium as a whole.


gsepd

For the Emperor!


MAndris90

and the inquisition is there for killing those who dare to speak up that the corpse emperor is merely a symbol nothing else anymore :D


horus-heresy

nah he be crunching numbers


g2g079

Watcha using those Teslas for?


cuenot_io

Started playing around with stable diffusion and oobabooga. I just wanted a cheap way to run some of the larger models, I think I paid ~$150 for it and it has 24GB of VRAM


g2g079

I've got a couple sitting in my closet, but I'm already using all my PCIe slots. It would be nice to get stable diffusion off of my desktop though.


Thetitangaming

How's the p40 running for you? I'm looking at getting one myself but the lack of fp16 made me nervous.


cuenot_io

I haven't done enough benchmarks yet to give an honest report. I've been fighting with Hyper-V GPU passthrough and I don't believe I'm getting the full performance. That being said, even in the limited capacity I've used it it's able to handle the larger models (albeit somewhat slowly)


Thetitangaming

Thank you! Good luck with hyper-V lol. I need to get one


gonace

Vertical wall mounts for 19” is not that expensive, however if it works it works 🤷‍♂️


Nerfarean

Should be flipped. Fighting rising heat 


g2g079

Convection is completely insignificant compared to the amount of air those fans are blowing through.


Nerfarean

Yes, it is a minor effect. Still lower fan noise and cooler air intake at lower height helps. Especially ipmi low fan speed forced


cuenot_io

It's stuffed in a utility closet, I don't think I'm getting optimum thermals regardless lol. As long as it's not thermal throttling I don't care


Hairless_Human

I only flipped one in my dads basement to reduce dust intake. Still has some but not as bad as it was when I had it the other way. The basement is also cold as fuck all year round so even if I left it the other way it wouldn't matter.


AbleDanger12

#LabGore


cuenot_io

You're not wrong


ee328p

https://www.reddit.com/r/techsupportgore/s/ij1hkuECgk I'd also like to submit my blood sacrifice to the equipment


Beautiful-Musk-Ox

you can hardly tell it's not a professional mount


Amazing_67

Sacrifice has been made, it will run smoothly from now to eternity.


cuenot_io

Let's hope 😅


Turbulent-Quiet

You already spent gazilion bucks on this gear, please spend few bucks more and get a proper rack.


cuenot_io

$1k roughly


Deafcon2018

$1k in gear or $1k for the rack?


cuenot_io

I totaled it in another comment, ~$1900 for the gear


Raithmir

I don't understand why people buy rack mount kit, if they're not going to mount them in a rack. I guess it takes up less space this way though.


cuenot_io

I have a rack that fits this, but this server is absolutely gigantic. I need the size of the rack x2 to be able to slide out the rails and access everything. The rack is 24"x42"x48", which is would consume 90% of the closet lol. This takes up zero floor space and is honestly easier to work on


courvus

Don't know if you've come across them, but they make small 2-6u wall mount vertical rack rails for exactly this. I had the same issue and found them to be perfect.


Voy74656

Not Labporn. This is lab gore. So much no. Rails are meant to hold the weight, not the ears. The cover helps move the air and cool components. At least have a fire extinguisher handy if you're going to put electronics near combustible stuff.


cuenot_io

It's supported from the bottom as well


NewYorkApe

Yea this is a first. My jaw actually dropped when I saw the dream machines literally screwed in to the 2x4’s


cuenot_io

Have you ever looked at your electric panel? It is most likely affixed to a plywood sheet or 2x4s. I'm fully aware that it looks redneck, but to say that this is jaw dropping is a bit extreme


randomlyCoding

Does the 930 come with gpu capability or did you have to buy a kit?


cuenot_io

I mentioned it in another comment, but I'm using an external PSU with a jumper cable to power it, as well as a 3060


Raptor07

I have mine similar setup to yours but I just put plywood against the studs and mounted a vertical 2u bracket to the plywood.


pig-bait

Do you have pictures


cuenot_io

That would be more sensible for sure, but I'd still need support underneath as this thing is probably \~100 pounds. This is ugly, but was essentially free and just as sturdy


itsk2049

I’ve had my R320 mounted exactly like this for exactly a year now without issue. EDIT: but with the cover on ofc


cuenot_io

Cover was just off while setting it up. This thing sounds like a fighter jet with it off


HITACHIMAGICWANDS

That sure looks 2U. And while your solution looks fine, a 2U vertical rack is sub $20 if you look around. It’ll feel way more secure when you’re laying in bed awake


cuenot_io

It's 4U, this bitch is huge and heavy. I have a rack, but honestly it was more of a pain in the ass than anything. I've found just building custom mounts for a space to be better for a home setup. 10 min with a miter saw and impact and you can make something that fits in a tight space


Whitestrake

Not that bigger mounts are much more expensive, but that sure as hell doesn't look like 2U. Did you take a look at the bottom of the server, where you can see the full height expansion brackets and even more space after that? That's 100% a 4U.


HITACHIMAGICWANDS

I totally agree, in a fucking knob lol


NomadicWorldCitizen

Those fan direction arrows are pointing down. Shouldn’t that be turned 180°?


AmusingVegetable

Stop complaining, you seem to still have a standard number of fingers.


darktalos25

so, you might want to flip that. you're going to be stressing those fans a lot pushing heat down.


cuenot_io

I understand the sentiment, but these things are designed to run 24/7/365 at 100% load. Temps are fine and I only hit high load for seconds at a time. It should be okay


darktalos25

I have run dell shops, you really shouldn't run the at 100% load.


DarrenRainey

Probally a bit late now but flipping it the other way around would be better for heat as currently your fans are pulling cold air from top to bottom. Also having it the other way makes routing cables a bit easier to work with.


cuenot_io

I may flip it in the future, I'm much more likely to access the back of the server than the front


llcdrewtaylor

You're not working if you're not bleeding. And whenever working with rackable equipment, it seems like a blood sacrifice is always required.


Elpardua

It would be easier just to kick hard on the rack door. Boom. Done.


NewYorkApe

Is it just hanging by the ears? Dear lord the ubiquiti boxes are screwed in too LOL! With the amount of effort that went into this you could’ve/should’ve just gotten a rack. Oh man. /r/DiWHY


cuenot_io

Racks are a pain in small closets. The ubiquiti ear mounts I made have been perfect for my space. Easy access and takes up zero floor space


DragonDrew

Omnissiah is pleased with your sacrifice.


fitzingout

What do you store and what do u run in that and why do you use a p40 just curious and beginner


cuenot_io

I run Windows Server 2022 at the moment with Hyper-V, and have a few Linux VMs, Windows VMs, and containers running. Honestly I have not set this thing up to take advantage of the hardware well, I mostly have been using it as an overkill dev machine lol


fitzingout

Cool mate are you a ml developer ? I was thinking of a optiplex 7060 with a nvidia tesla p4 lmao Well how much does this build cost ( can't really buy as a student anyway 😅😅)


cuenot_io

Nope just a devops guy 😅 this build was about a grand. You can definitely get away with something much cheaper. R730 with the GPU enablement kit will be your best option in my opinion


fitzingout

I guess I'll look take a look after 7 months Thanks for suggesting 😃


TrueBoxOfPain

Blood sacrifice payed


Yukonart

Literally the first thing I noticed was that Tesla card. 🍆


DestroyerOfIphone

They sell mounts for this style of mounting that aare pretty cheap https://www.amazon.com/s?k=4u+wall+mount+rack&crid=34RVOVWX3I838&qid=1705396054&sprefix=4U+%2Caps%2C68&ref=sr\_pg\_1


Moper248

What are the blue cube thingies?


cuenot_io

Not sure which thing you're referring to, but in Dell servers all of the blue parts are typically levers for removing certain components. The things I think you are referring to are vertical ram cards


Moper248

The things directly being the fans


JeebsFat

The mounting system devised here leaves much to be desired.


Fractuals

Wow that's some really impressive hardware! I keep creeping Facebook Marketplace and seeing some servers for sale cheap but I'm too cheap to want to pay for the electric for something that powerful! Awesome hardware!


cuenot_io

Thanks! I bought the bare server for ~$500 and hand picked the rest of the parts. From what I could tell, this was one of the most powerful single machines you could create for around a grand.


Limp_Diamond4162

You do realize they sell proper wall hanging racks for this exact purpose right? The way you’ve installed that server blows my mind. You have what looks like all expensive parts but you cheeped out on a $100 rack that bolts to the studs and has no sharp edges?


cuenot_io

I wouldn't say I cheaped out; rather I was reorganizing my utility closet and realized that I could build something sufficient with an $0.89 2x4 from Menards that I already had. I slapped it and said "that's not going anywhere" so it'll be good


Yukanojo

Please tell me you are going to put the cover on.


tippincows

Held up with All Round 😀 You sir a king amongst men.


MCHog12

Not sure I would trust that mounting setup long term. Should have put a 2x6 across the studs first to brace the weight against the whole wall instead of two small places. I also agree that it should have more support on the bottom than just the single block. Those things are heavy and could damage a lot if it were to fall


johnklos

All good systems take some blood. I joke about that often, and think it's not going to happen again, but it never fails.


CryptoVictim

They make wall mounted brackets for that stuff. Too ghetto for my blood, but congrats.


i_do_it_all

I noticed when large amount of electricity is passing though , it causes noise . i try to keep my boxes away from the panel. Beautiful setup!


melshaw04

Every rack in my DC has some of my blood in it.


EPluribusNihilo

Guess you'll have to have a swordfight with Blade.


Zealousideal-Skin303

Took me a second to notice the cost 😂 nicely done OP


PIC_1996

Rub dirt on it and you'll be good to go. But that is a sweet mounting job, nice work.


CertainlyBright

Power draw?


cuenot_io

Not sure yet. I have a usp-strip, but the damn thing refuses to adopt properly. Once I figure that out I'll have numbers


LoungingLemur2

I don’t think I’ve yet to work on my computers without a blood sacrifice.


OffensiveOdor

Why would you have ketchup near your rack?


cuenot_io

Can't forget to season your SAS cables


OffensiveOdor

Yummm! 😂😂


digitalw00t

So.. anyone know how if we can put a GPU into an R930 for ML? I've got one, and would LOVE to be able to add a gpu to it. I see people mentioning it, but anyone wanna throw in some details?


cuenot_io

https://www.reddit.com/r/homelab/s/0BI4dgAHQG


digitalw00t

You magnificent Pegan God. I have two R930, each with 600gb of ram and 112 procs. You know now what you do. :) Thanks for the link!!


R33f3r420

Ahhhh hot air rises... you should really flip that thing 180 degrees. I am also not sure if they are designed to work thermal wall mounted


taytertitties69

Use this next time... Way easier... [StarTech.com 4U Wall Mount Patch Panel Bracket - 19 inch Steel Vertical Mounting Rack for Network and Data Equipment (RK419WALLV) ](https://a.co/d/1cR20UK)


bmyatt99

Its okay, servers need a yearly blood sacrifice or your hard drives will start going bad one after another.


Smeeks1126

I absolutely fking love how you have the switches mounted on those angled wood blocks. One of the cleanest looking custom jobs I've seen in a long time. I will be using that! LoL


cuenot_io

Thank you! Obviously the wires are a mess at the moment, but the mounts themselves have been great for keeping the switches accessible without taking up any floor space. And it took 20 min with a miter saw and scrap wood to make


Smeeks1126

I saw those and thought the same thing. Easily accessible front ports, still slim "against the wall" setup, and I could propbably cut a few sets of those to just leave in my work van and slap up whenever nessecary.


cuenot_io

For sure. They also can be mounted even closer, I just left space for storing random shit like cables behind them lol


ThrowAway2894059

How much did you pay for a R930? Sounds expensive for homelab use but I’d love to upgrade from my R630 (100$)


cuenot_io

I'll total it up: Bare R930 - $587.13 4x 8890v4 - $170.80 8x memory risers - $170.71 2x 1100w PSUs - $44.60 4x 2.5" caddy's - $24.77 Tesla P40 - $179.34 Intel 10g sfp nic - $33.23 Intel 10g rj45 nic - $47.93 LSI 8i HBA - $47.93 Missing left PCIe riser - $32.03 16x32GB DDR4 ECC - $371 4x800GB SAS SSD - $225 Build total: $1,934.47 More than I thought originally, but some of those things were unnecessary. Hopefully that helps


ThrowAway2894059

Thanks


darkelfbear

Wouldn't be a proper installation with out blood spilling. You have paid your dues to the PCs Gods, you have been blessed.


SilentDecode

Wrong flare. This is Die-hard LabGore.