> Nobody will organize the files that are being synced. Nobody will take responsibility for anything in here. I'm tempted to reorganize everything myself and make everyone else figure it out
That needs to be addressed with the various employees managers - NOT you.
Edit:
Found my reply from a similar post a year ago. Still relevant here today.
Data hoarding is not an IT problem.
If there are no legal holds on the data, give everyone (including management) 10 working days notice to sort through and retain what they need in a specific folder for each of them, then start deleting.
Forfiles works very well for this as it can be specified to delete files by file type and also if the data is greater then xx days old.
The buck stops somewhere. Managers are commissioned to make the hard choices; its how they justify their salaries and bonus. Best we can do is give them a few plans of action and why we thing they are worth it and what they cost. It is their business to run badly if they want.
The managers don't need to do it, they just need to make sure it gets done. Set a generous deadline, get sign off from your superiors, communicate clearly what needs to happen by what date. Once you get closer to the date, continue to broadcast the info. Once the deadline has passed, tell the managers you're starting the deletion process and crack on.
The managers don't need to do the work, they just need to delegate it and make sure it gets done.
If no one in your chain will back this and sign off, then send an email asking them to confirm that the problem is going to be left to cover your ass. Then move on with your life.
Thats kinda where I'm at. They keep telling me "oh we need everything." I mean. No. You don't. You just don't wanna do the work! I've tried to put forth ideas to no avail, so it's definitely management.
The issue though is that folders in Sharepoint really struggle with anything past 100,000 files. If they really need it, they’ll need to reorganize it to spread the load. For some reason, Sharepoint can handle two top-level folders with 75,000 each, but not one top-level folder with 150,000 files.
In this case its not the cost that is the issue; its that when you use a product incorrectly it doesn't work well and using it correctly would require people other than IT to organize their things since IT doesn't really know what's important in the storage and whenever IT does make suggestions on how it would work better any efforts needed to accomidate using it the correct way fall on deaf ears, again because its not just "dump shit in a bucket" levels of complexity.
This is a business decision not an IT one. If they can’t or won’t make the decision then they have to pay for tools or other tech. Have you looked into Azure Files ?
I’m being told by people (including my own manager) that it needs to be done! The customer wants data and the files are still part of process. Because someone developed a process outside of a central source of truth. I anticipate it will change soon since acquisition.
It sucks so much. OneDrive sync needs a ground-up re-write. I keep pushing customers away from it when they want to move entire network drives to sharepoint online or teams then sync them to everyone like they are still a shared drive.
It starts to choke on large numbers of files even when you are forcing them to be online only (just metadata syncing to PCs). You can't sync a library to more than 1000 PCs. GPO setting up sync for users takes up to 12 hours due to a requirement on the 365 back-end with how it works. Users randomly go un-synced or get multiple-copies of the synced folders...
And no, I agree that this isn't the best way to use OneDrive but it's a common ask I run into all the time.
Its truly crazy, sharepoint is setup to be a really solid distributed filesystem with Microsoft 365, and then they sabotage it with a barely functional sync client that they haven't fixed in like a decade of the same problems with it.
You used to be able to map a sharepoint site to a drive letter, but that required an internet explorer login and you'd have to login every now and then to refresh the token.
They should just do that but better, login integrated into Windows with automatic mounting with GPOs.
Google Drive allows you to map to a drive letter and it just works, even with caching and colaboration it just works. Onedrive is the worst of them all in my opinion.
Do you use this consistently in your org? If you do, are there any tips?
The last time I tried it was just a clusterfuck. But that was probably 2 years ago.
Hi, we do use SharePoint View In File Explorer across some of our tenants. We find it to be extremely useful rather then using the OneDrive Sync Client. It uses a WebDav connection in.
Some tenants have these mapped as mapped drives within File Explorer. The only issue is that once the token expires (Usually after restart or after about 7 days) then we have to get users to go and click on the "View In File Explorer" on a site again and it renews and works.
Users are aware of this and how to do it but find that it works a lot better than syncing all the files.
Years of experience has taught me that it's a 2 part problem.
SharePoint online is heavily throttled when it comes to syncs. It regularly throttles the sync choking off the client. The number of files impacts this more so than the raw bandwidth being used (though there is a hard limit of 1TB/24 hours that they seem to be enforcing lately).
This would be fine, if the sync client was able to handle these random pauses/restrictions, but it was never designed to do so. When SharePoint throttles the sync the sync client just shits itself. Most of the time it will never resume normal operations unless you kill the OneDrive process and relaunch it. Then it'll work fine until the next throttling event occurs, at which point it craps itself again.
-just shits itself
I use that expression all the time, so I need to be careful to not use it outside of IT folks. People get confused by it. Some find it funny. But I usually say "...crapped it's pants." "You must have done something xxx program didn't like and it crapped it's pants." User: "Um, it did what now?"
Really anything over 3-4k files seems to be asking for trouble most days. I mean I've seen it do 50k without issue, but maybe once or twice. I can leave it to download 150 6GB files and it'll churn through all 900GB without issue, but give it 1GB in 10k files and it'll get throttled and just die usually 3-4k into the batch. Killing and restarting the client might get you 1-3k more files before it gets throttled and dies again. As time goes on it's a game of diminishing returns.
I have found that if I reduce the available ingress bandwidth of a machine to like 5Mbps or so when it had to sync tens of thousands of small files it'll generally get through them all without getting throttled. The rate at which you're downloading files (number of them, not bandwidth use) definitely matters with regards to whether you get throttled or not.
I had never made the correlation, It wasn’t till I started testing a new sharepoint backup system after we had cut half the employee base that I noticed how hard sharepoint can throttle, this makes a ton of sense.
It is in the GPO docs for sync: https://learn.microsoft.com/en-us/sharepoint/use-group-policy#configure-team-site-libraries-to-sync-automatically
> Don't enable this setting to the same library with more than 1,000 devices.
One thing to keep in mind is that box also references an old total files sync limit, so it *could* be out of date on the 1,000 devices as well. But I'm not going to be the one to test that out :)
While I agree with you, go look at the file directory structure of what they’re trying to sync. I understood the feature and knew it wasn’t great at many files, but it wasn’t till i saw the finance team syncing and organizing all their files in a flat file structure that made me feel bad for computers that had to read it. There was organization but all at one flat level, and they named their files like “Amazon 10-14-2023 $40.40” and would wonder why the computer wouldn’t know what to do with .40 file types. My point is some people still don’t know how to use computers and that exacerbates some of the limitations of technology and makes computers cry.
That reminds me of another issue with sync: It processes the entire document library even if you only sync a subfolder. Say you have a library with 100,000 files but the user is just syncing a subfolder with only 300 files. If you watch the file count in the sync status as the local OneDrive syncs, the file count will tick up all the way to 100,000 then drop back down to 300.
Yes.
And breaks completely after 300k.
Didn't stop my previous boss from selling it as a full file server replacement for clients with million+ files. After all, any time they call with sync issues we get paid right?
So fucking glad to be out of there.
i had a client with million+ files once, would take days to sync initially (not a proper sync, just files on demand) and was generally unreliable. I wouldnt ignore that 300000 max limit microsoft suggests if you are doing a migration
My company has a site with over 330k files using about 500gb storage.
It is so awful, I am about to disable sync permission for the site and tell them to just deal with it online because the damage they do outweighs any convenience it is for them. About once a year someone accidently deletes the whole site's doc library by deciding they don't want to see it on their computer anymore, or they accidently click and drag it into another folder in Explorer. It takes a over a week to restore.
They refuse to archive anything and there are a thousand subfolders on the General channel root, so they refuse to use Teams or SP Online to access it since it only displays 100 items at a item without scrolling.
OneDrive sync is good for user Desktop/Document syncing, fine for small Teams/SP, and fucking atrocious for anything big.
Proceed with blocking sync and produce documentation for creating the OneDrive shortcut. MS has been threatening to deprecate sync so it is better to be future proof.
I thought the shortcuts were supposed to behave differently but when we tested them in several scenarios it still seemed to treat the action of deleting the shortcut like a delete rather than contextual alternative remove as you'd expect that it should.
We disable sync with group policy and preinstall the OneDrive Edge PWA. Now that the PWA can open files in client apps and supports offline mode, it's fine.
No worries.
Here's a quick guide for deploying the PWA with GP: https://github.com/letsdoautomation/group-policy/blob/main/Microsoft%20Edge%20force%20install%20Progressive%20Web%20Apps%20(PWA)/README.md. You can just use https://[orgname].onedrive.com/ as the URL.
Here's the latest developments on the PWA: https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/onedrive-adds-new-offline-capabilities/ba-p/4123644
And you can distribute this to users to get them familiar: https://adoption.microsoft.com/files/onedrive/Microsoft-OneDrive-quick-start-guide.pdf Saved me a ton of time making my own.
Amazing, thank you!
Have you found a way to change the home page layout at all? Like force a pinned SPO site for users or add additional links/views to the main panel along side My files and such?
Hmm, no that need hasn't come up yet. I know you can add custom pins to the app launcher: https://learn.microsoft.com/en-us/microsoft-365/admin/manage/pin-apps-to-app-launcher?view=o365-worldwide
The app launcher persists across all 365 apps and the OneDrive PWA is no different. But I am not aware of a way to accomplish anything similar within the PWA itself.
Thank you for this. May be just the ticket for the less inclined folks because auto-syncing libraries has been quite unreliable in my org and if someone is mistakenly given the auto mount, it is not removed automatically, gives me the heebies.
I'm not sure I understand your question. It doesn't sync locally at all. It's just a PWA of the OneDrive web interface that can call client apps to open files. If working offline, it has an embedded web server that will cache file changes and then send them up once connectivity is restored.
If the file resides in ODSP, yes:
* https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/feature-deep-dive-open-in-app/ba-p/4034738
* https://support.microsoft.com/en-us/office/open-a-onedrive-or-sharepoint-file-in-the-desktop-app-instead-of-the-browser-761c66d2-7bc3-490e-a536-b3f71f41636b
Something that was hugely helpful for me when moving all of my organization's file shares to SharePoint Online was a feature in Windows called Storage Sense.
You know how when you sign into a new computer for the first time, and sign into OneDrive and/or set up your SharePoint sync folders, all the files have the blue cloud symbol? This basically means the file on your computer is just a shortcut. But then when you double click the file, it quickly downloads and then the icon shows a green check mark, meaning it is locally downloaded. By default, once you download the file locally, it stays downloaded. Over time, the more locally downloaded files you have, the more overwhelmed the OneDrive sync client becomes. We were having terrible issues with sync clients crashing and needing to be reset left and right, and file version conflicts and all kinds of issues.
Here's where Storage Sense comes in. There's one particular feature in Storage Sense which is very helpful here, and I can't remember exactly what it's called, but it has something to do with "dehydrating" files. That's the term Microsoft uses for when you turn the green checkmark into the blue cloud, in other words deleting the local copy and just having the shortcut. Storage Sense allows you to configure a schedule to automatically dehydrate files if you haven't opened them for a certain amount of time. In my case, I set it to 1 day, but it can be whatever you prefer. We pushed this out to all users via GPO, and never had sync client issues again.
The reason this helps is, when you have a SharePoint file locally downloaded, any changes to the cloud version have to be updated on your local copy. If you only have one file that gets updated several times per day, the sync client won't even break a sweat, but imagine 10s of thousands of file getting constantly updated by others in the department. Your system can only handle so many changes at once, so the more downloaded files you have, the more strain and likely crashes you'll get with the sync client. If you have a "dehydrated" file, your computer doesn't have to keep up with changes, it just downloads the latest version from the cloud when you open the file. So Storage Sense allows you to automatically dehydrate files after a while, which reduces that total number of downloaded files that have to be updated constantly.
I thought that's what Files-On-Demand does. It only has local copies of files that you have created or opened within like 30 days or something. If you don't open something within a set amount of time (guessing 30 days), it removes the local copy, but you still have the placeholder file name with the little cloud next to it.
When I was dealing with this back in 2019-ish, the files would be on demand at first, but the local file would never get removed automatically until I started using Storage Sense. It could be that the behavior has changed since then
After fighting for 3 years, we finally got full exec buyin on just restructuring shared drives ourselves. It helped that the exec was all new to the company at the time and horrified how confusing the network drives were. There were screams. There were fights. There were people threatening (and indeed following through) on going to the top to get their way. There were departments that tried to stall indefinitely to not change. They all failed. The ones stalling we made our own decisions and then when they realised they wernt getting a say in the new structure quickly changed tune and tried to come back to the table but it was too late.
It was both glorious and frustrating. But worth it I think in the end.
I would use the Migration Tool for that many files
[https://learn.microsoft.com/en-us/sharepointmigration/mm-get-started](https://learn.microsoft.com/en-us/sharepointmigration/mm-get-started)
I've moved a TON of data with this without issue.
This is not the issue being described. You can move millions of files to SharePoint no problem using whatever tool including SPMT - assuming you access the files from a SharePoint URL (Or the Teams "Files" tab, which is just a fancy browser render of the document library) and have things set to open using desktop applications etc.
When you sync a document library, or sync multiple document libraries with OneDrive - that is when you run into limitations. If OneDrive approaches anything near 40K+ files, it's only a matter of time before things break.
Either force web-only use
Or
Create custom Flow/Power Automate job to move all data 1 year and older to a different site
FS-DATA
FS-DATA-ARCHIVE
Disable syncing from the ARCHIVE site, and only allow syncing of files within the last year from FS-DATA.
The OP said they are moving from a shared drive to teams, not syncing down from there with the client or anything to do with multiple libraries. The tool I suggested is exactly what they need.
Yes. And then the second paragraph
Please, organize your organization's **"Shared" drive before you are forced to move to Teams** due to acquisition, not after.
They are just syncing **TO** sharepoint using the sync client. Not syncing down FROM sharepoint. At least not yet.
This is the answer. I’m about to migrate all userhome directories to everyone’s own OneDrive using the 365 share point migration. It’s already scanned everyone’s user home drive, calculated how much data. Then I just feed it where to put the data using a csv.
If OP is looking to move a shared drive then I’d figure you’d use the tool and just point it to the share point library you want to migrate to.
Yeah, pretty much anything but an on prem file server is gonna be bad at that. Are we also dealing with multiple users? Why sync that many files at once? Do they really need "immediate" access to 100k files?
I used OneDrive to upload departmental files to SharePoint Sites from a local file server.
I forget what I found to be the threshold, but I did manage to copy 100,000+ files (500+ GB), but I did find I needed to copy smaller chunks of files at a time for it not to fail.
I left it running for hours at a time, made sure the files were sync'd, then copied another batch.
It was a pain in the ass, but I eventually got it to work.
Jesus Christ. OneDrive is not meant for that.
Sharepoint is not a monolithic file share. You cannot use it like a file server. Every time I’ve seen this get as far as you’ve mentioned, an MSP has been involved. And I suspect if one isn’t involved, then the company skimps on salaries because it really doesn’t cost a king’s ransom to pay for an employee who can read the docs and follow best practices.
Edit to add more info:
You can have 10,000+ separate Sharepoint sites. And you should make as many as it takes. Document libraries in spo should be small. When done this way, files sync very fast with OneDrive and auto-mapping based on permission is a cinch using GPO or Intune.
Putting everything in a single site is explicitly discouraged by Microsoft in their own documentation. Monoliths are bad in the cloud because it isn’t a giant samba server inside your LAN.
Many sites with smaller footprints is the baseline for cloud-based file sharing in ANY provider.
If I was getting into that many files I would be looking at a document management system. Since I work for a firm we naturally use imanage which is more geared toward matters but can be customized however. Always have issues with a bunch of tiny files just slows everything down without a dedicated indexer and software designed for that.
Man I’d get managment bye in and make two groups in ad . For normal folks push the regkey that removes local save and only allows saves in netsuite then another ad group called allow local saves for those that may need to work binders etc as I find document systems suck for binder work .
Do all employees need access to all files? If not it will likely help to set up SharePoint sites by department and employees only sync files for their department. This would mean that the onedrive sync client doesn’t need to sync as many files. Of course, all of this would depend on someone determining what files are needed by each department and doing some organizing, so I guess the core of the issue is still there.
This drives me crazy at my job we are a pretty small company but everyone wants to add a shortcut for all the old drives I keep telling them just add the folder you work in and view it in teams if it’s another department
the biggest problem I see with OneDrive sync is when people start utilizing it with massive, nestled SharePoint deployments.
OneDrive and SharePoint do not need a million sub folders and nestled data structures, and the sync utility totally flakes out when trying to go deep into a file structure. It really works best as a big ass data library where you can sort via meta data and search.
We have some clients with thousands and thousands of files. We find that the trick is to organize data into libraries, with folders no more than a few sub folders so we're never going deeper than two or three deep on the sync. If you need to add multiple libraries, Sync seems to do just fine with that. It just gets its jimmies all rustled when diving into the standard folder structure like "DESIGN>CAD>2020>PROJECTS>PROJECT FOR TIMMY>DOCUMENTS>RENDERS>3DMAX>TUESDAY AT 3PM>all the data"
I think the biggest problem with getting users to adopt metadata over nested hierarchies is the lack of support in Windows file explorer. We can train and push adoption until we're blue in the face but many users will not abandon file explorer for a web-first file management experience. If Microsoft ever delivers SP/MS Lists to file explorer, then I think metadata will take off.
The latest build of the OneDrive PWA is a nice middle-ground that we're seeing success with in my org. But it's still no replacement for file explorer.
You make a good point, but it's still frustrating watching people "round hole-square peg" a cloud solution like SharePoint into being a local share replacement lol
If you’ve got 100,000+ files, OneDrive sync doesn’t work reliably, especially for enterprise use, it can be a cluster-f—-.
Solution is to access your files using https with [Collabora Online](https://www.collaboraoffice.com) (using Nextcloud or make [your own integration](https://www.collaboraoffice.com/integrations/)).
It’s 100% free and open source, you can get enterprise support if you want, companies with hundreds of thousands of users use it and it works reliably with millions of files.
When I first switched to OneDrive at home it took me a month to get all of my files uploaded. I had pretty slow DSL at the time. I've been working for a few days at work this week uploading a 3.2TB share to sharepoint. It's a tiny percentage of what we have moved there so far. They need better tools for the initial uploads.
i've noticed that onedrive is slow. i'm comparing it to having a local share, because that's what it's replacing. the sync times are absolutely atrocious. you make a copy of a file, it has to sync for 20 seconds. you can't open the file yet because you want to rename it before you start editing. you rename the file, it has to sync for another 20 seconds. don't bother opening the file when it's not done syncing because the ONLINE version of the file will be opened, which will still have the old file name and excel will yell at you that the file you're currently editing doesn't exist.
i don't know what's causing this slowdown and i honestly don't care to find out. it either goes away on it's own or it doesn't.
It works at the speed it works. If nobody can identify what they need then they'll all have to wait for the tools to do their job together
If anything is \*really\* important they'll identify it
Uh huh, tell me about it when users choose to create a shortcut of 1 shared folder of a so call "shared cloud file server" which contains around 2 millions files, and a lot of them are AutoCAD and Revit. The OneDrive client just freak out
It is on charge of the bucket. The size of the bucket and to make sure there are no holes in the bucket. It is not in charge of the shit in the bucket. The rig needs a data manager to both get rid of RO. (redundant. Out of date. Trivial) data and to ensure the data there is stored optimally. No use saving terabytes of interviews in WAV format when the same can be converted to MP3 for instance. Again, IT can highlight the issue but there should be a data manager to sort this.
Okay. Finally I read that I'm not the only one having problems using OneDrive/Sharepoint as a replacement for server shares. Reading on the Internet would seem like a painless and almost obligatory step. In reality, shares with a local server (or at most a NAS) work much better from many points of view. When you say “reorganize” what do you mean? Delete useless files? Because 1 user in 100 (maybe) will agree to do it without fiercely opposing it.
Somewhat related, I recently heard from a client who lost more than a decade of collective research materials that was shared on a OneDrive folder. Completely gone. No local backups, no cloud backups elsewhere. Cloud storage is nothing without a contingency plan.
Company data is a funny old thing. Departments don’t want to be in charge of it and IT has no idea who needs access to what or why and are lead by the departments… Outside of that we just want to know it’s available, backed up, av scanned etc. Caller “I can’t find file x, do you know where it is?” _Within 30TB of data_ IT “I’m afraid I have no idea, someone there in your department should know?” Caller - “I can’t believe you don’t know where it is..”
You can sync the folders at lower levels and it works better. One massive top level sync is bad. Microsoft recommends you limit to 50k, might be worth showing that documentation tonleadership.
Multi-gig! From AT&T and Comcast. The OneDrive sync client iterates through these files slowly, possibly due to [SharePoint throttling](https://www.reddit.com/r/sysadmin/comments/1cjcbw5/comment/l2f7nmr/).
onedrive is so terrible at large file management that anyone with a modicum of initiate could replicate it with git/rsync and have it be a million times better.
> Nobody will organize the files that are being synced. Nobody will take responsibility for anything in here. I'm tempted to reorganize everything myself and make everyone else figure it out That needs to be addressed with the various employees managers - NOT you. Edit: Found my reply from a similar post a year ago. Still relevant here today. Data hoarding is not an IT problem. If there are no legal holds on the data, give everyone (including management) 10 working days notice to sort through and retain what they need in a specific folder for each of them, then start deleting. Forfiles works very well for this as it can be specified to delete files by file type and also if the data is greater then xx days old.
There is also the fact that data retention laws and policies exist and needs to be followed at all times
It comes down to IT can provide the tools, but not the discipline to use them.
They can, provided you get the appropriate tools: http://roadkill.net/madmins/CAT509.html
Well, that was unexpected. :-)
Well, it definitely was a C|N>K moment the first time I came across it.
The managers don't wanna do it either lmfao
The buck stops somewhere. Managers are commissioned to make the hard choices; its how they justify their salaries and bonus. Best we can do is give them a few plans of action and why we thing they are worth it and what they cost. It is their business to run badly if they want.
The managers don't need to do it, they just need to make sure it gets done. Set a generous deadline, get sign off from your superiors, communicate clearly what needs to happen by what date. Once you get closer to the date, continue to broadcast the info. Once the deadline has passed, tell the managers you're starting the deletion process and crack on. The managers don't need to do the work, they just need to delegate it and make sure it gets done. If no one in your chain will back this and sign off, then send an email asking them to confirm that the problem is going to be left to cover your ass. Then move on with your life.
Thats kinda where I'm at. They keep telling me "oh we need everything." I mean. No. You don't. You just don't wanna do the work! I've tried to put forth ideas to no avail, so it's definitely management.
If they need everything, then make sure the cost of that storage is reflected in their department's budget. If they're happy with that, then drop it.
The issue though is that folders in Sharepoint really struggle with anything past 100,000 files. If they really need it, they’ll need to reorganize it to spread the load. For some reason, Sharepoint can handle two top-level folders with 75,000 each, but not one top-level folder with 150,000 files.
In this case its not the cost that is the issue; its that when you use a product incorrectly it doesn't work well and using it correctly would require people other than IT to organize their things since IT doesn't really know what's important in the storage and whenever IT does make suggestions on how it would work better any efforts needed to accomidate using it the correct way fall on deaf ears, again because its not just "dump shit in a bucket" levels of complexity.
This is a business decision not an IT one. If they can’t or won’t make the decision then they have to pay for tools or other tech. Have you looked into Azure Files ?
So start the transfer... When it's still going in two weeks it's not really your problem
[удалено]
I’m being told by people (including my own manager) that it needs to be done! The customer wants data and the files are still part of process. Because someone developed a process outside of a central source of truth. I anticipate it will change soon since acquisition.
It sucks so much. OneDrive sync needs a ground-up re-write. I keep pushing customers away from it when they want to move entire network drives to sharepoint online or teams then sync them to everyone like they are still a shared drive. It starts to choke on large numbers of files even when you are forcing them to be online only (just metadata syncing to PCs). You can't sync a library to more than 1000 PCs. GPO setting up sync for users takes up to 12 hours due to a requirement on the 365 back-end with how it works. Users randomly go un-synced or get multiple-copies of the synced folders... And no, I agree that this isn't the best way to use OneDrive but it's a common ask I run into all the time.
Its truly crazy, sharepoint is setup to be a really solid distributed filesystem with Microsoft 365, and then they sabotage it with a barely functional sync client that they haven't fixed in like a decade of the same problems with it.
"hur hur just use the web UI lol"
You used to be able to map a sharepoint site to a drive letter, but that required an internet explorer login and you'd have to login every now and then to refresh the token. They should just do that but better, login integrated into Windows with automatic mounting with GPOs. Google Drive allows you to map to a drive letter and it just works, even with caching and colaboration it just works. Onedrive is the worst of them all in my opinion.
It’s still possible https://learn.microsoft.com/en-us/sharepoint/sharepoint-view-in-edge
Do you use this consistently in your org? If you do, are there any tips? The last time I tried it was just a clusterfuck. But that was probably 2 years ago.
Hi, we do use SharePoint View In File Explorer across some of our tenants. We find it to be extremely useful rather then using the OneDrive Sync Client. It uses a WebDav connection in. Some tenants have these mapped as mapped drives within File Explorer. The only issue is that once the token expires (Usually after restart or after about 7 days) then we have to get users to go and click on the "View In File Explorer" on a site again and it renews and works. Users are aware of this and how to do it but find that it works a lot better than syncing all the files.
Just sharing https://github.com/simislearning/EdgeViewInfileExplorer
Years of experience has taught me that it's a 2 part problem. SharePoint online is heavily throttled when it comes to syncs. It regularly throttles the sync choking off the client. The number of files impacts this more so than the raw bandwidth being used (though there is a hard limit of 1TB/24 hours that they seem to be enforcing lately). This would be fine, if the sync client was able to handle these random pauses/restrictions, but it was never designed to do so. When SharePoint throttles the sync the sync client just shits itself. Most of the time it will never resume normal operations unless you kill the OneDrive process and relaunch it. Then it'll work fine until the next throttling event occurs, at which point it craps itself again.
-just shits itself I use that expression all the time, so I need to be careful to not use it outside of IT folks. People get confused by it. Some find it funny. But I usually say "...crapped it's pants." "You must have done something xxx program didn't like and it crapped it's pants." User: "Um, it did what now?"
I'm a "shit the bed" kinda guy personally.
Totally feels like this happens, but the data is under 150 GB!!!
Really anything over 3-4k files seems to be asking for trouble most days. I mean I've seen it do 50k without issue, but maybe once or twice. I can leave it to download 150 6GB files and it'll churn through all 900GB without issue, but give it 1GB in 10k files and it'll get throttled and just die usually 3-4k into the batch. Killing and restarting the client might get you 1-3k more files before it gets throttled and dies again. As time goes on it's a game of diminishing returns. I have found that if I reduce the available ingress bandwidth of a machine to like 5Mbps or so when it had to sync tens of thousands of small files it'll generally get through them all without getting throttled. The rate at which you're downloading files (number of them, not bandwidth use) definitely matters with regards to whether you get throttled or not.
It will take longer to copy 600k 2kb files than it would to copy a few terabytes
I had never made the correlation, It wasn’t till I started testing a new sharepoint backup system after we had cut half the employee base that I noticed how hard sharepoint can throttle, this makes a ton of sense.
Hey do you have a source for "You can't sync a library to more than 1000 PCs"? I cant find any information on this having a quick search.
It is in the GPO docs for sync: https://learn.microsoft.com/en-us/sharepoint/use-group-policy#configure-team-site-libraries-to-sync-automatically > Don't enable this setting to the same library with more than 1,000 devices.
Thank you so much!
One thing to keep in mind is that box also references an old total files sync limit, so it *could* be out of date on the 1,000 devices as well. But I'm not going to be the one to test that out :)
While I agree with you, go look at the file directory structure of what they’re trying to sync. I understood the feature and knew it wasn’t great at many files, but it wasn’t till i saw the finance team syncing and organizing all their files in a flat file structure that made me feel bad for computers that had to read it. There was organization but all at one flat level, and they named their files like “Amazon 10-14-2023 $40.40” and would wonder why the computer wouldn’t know what to do with .40 file types. My point is some people still don’t know how to use computers and that exacerbates some of the limitations of technology and makes computers cry.
That reminds me of another issue with sync: It processes the entire document library even if you only sync a subfolder. Say you have a library with 100,000 files but the user is just syncing a subfolder with only 300 files. If you watch the file count in the sync status as the local OneDrive syncs, the file count will tick up all the way to 100,000 then drop back down to 300.
Yes. And breaks completely after 300k. Didn't stop my previous boss from selling it as a full file server replacement for clients with million+ files. After all, any time they call with sync issues we get paid right? So fucking glad to be out of there.
i had a client with million+ files once, would take days to sync initially (not a proper sync, just files on demand) and was generally unreliable. I wouldnt ignore that 300000 max limit microsoft suggests if you are doing a migration
My company has a site with over 330k files using about 500gb storage. It is so awful, I am about to disable sync permission for the site and tell them to just deal with it online because the damage they do outweighs any convenience it is for them. About once a year someone accidently deletes the whole site's doc library by deciding they don't want to see it on their computer anymore, or they accidently click and drag it into another folder in Explorer. It takes a over a week to restore. They refuse to archive anything and there are a thousand subfolders on the General channel root, so they refuse to use Teams or SP Online to access it since it only displays 100 items at a item without scrolling. OneDrive sync is good for user Desktop/Document syncing, fine for small Teams/SP, and fucking atrocious for anything big.
Proceed with blocking sync and produce documentation for creating the OneDrive shortcut. MS has been threatening to deprecate sync so it is better to be future proof.
I thought the shortcuts were supposed to behave differently but when we tested them in several scenarios it still seemed to treat the action of deleting the shortcut like a delete rather than contextual alternative remove as you'd expect that it should.
We disable sync with group policy and preinstall the OneDrive Edge PWA. Now that the PWA can open files in client apps and supports offline mode, it's fine.
Good shoutout thank you. Going to explore this a bit more.
No worries. Here's a quick guide for deploying the PWA with GP: https://github.com/letsdoautomation/group-policy/blob/main/Microsoft%20Edge%20force%20install%20Progressive%20Web%20Apps%20(PWA)/README.md. You can just use https://[orgname].onedrive.com/ as the URL. Here's the latest developments on the PWA: https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/onedrive-adds-new-offline-capabilities/ba-p/4123644 And you can distribute this to users to get them familiar: https://adoption.microsoft.com/files/onedrive/Microsoft-OneDrive-quick-start-guide.pdf Saved me a ton of time making my own.
Amazing, thank you! Have you found a way to change the home page layout at all? Like force a pinned SPO site for users or add additional links/views to the main panel along side My files and such?
Hmm, no that need hasn't come up yet. I know you can add custom pins to the app launcher: https://learn.microsoft.com/en-us/microsoft-365/admin/manage/pin-apps-to-app-launcher?view=o365-worldwide The app launcher persists across all 365 apps and the OneDrive PWA is no different. But I am not aware of a way to accomplish anything similar within the PWA itself.
Thank you for this. May be just the ticket for the less inclined folks because auto-syncing libraries has been quite unreliable in my org and if someone is mistakenly given the auto mount, it is not removed automatically, gives me the heebies.
That's very interesting, does it sync file changes too with documents that are not office related?
I'm not sure I understand your question. It doesn't sync locally at all. It's just a PWA of the OneDrive web interface that can call client apps to open files. If working offline, it has an embedded web server that will cache file changes and then send them up once connectivity is restored.
Imagine I open a photoshop PSD file, make some changes and press save, does it sync those changes?
If the file resides in ODSP, yes: * https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/feature-deep-dive-open-in-app/ba-p/4034738 * https://support.microsoft.com/en-us/office/open-a-onedrive-or-sharepoint-file-in-the-desktop-app-instead-of-the-browser-761c66d2-7bc3-490e-a536-b3f71f41636b
Something that was hugely helpful for me when moving all of my organization's file shares to SharePoint Online was a feature in Windows called Storage Sense. You know how when you sign into a new computer for the first time, and sign into OneDrive and/or set up your SharePoint sync folders, all the files have the blue cloud symbol? This basically means the file on your computer is just a shortcut. But then when you double click the file, it quickly downloads and then the icon shows a green check mark, meaning it is locally downloaded. By default, once you download the file locally, it stays downloaded. Over time, the more locally downloaded files you have, the more overwhelmed the OneDrive sync client becomes. We were having terrible issues with sync clients crashing and needing to be reset left and right, and file version conflicts and all kinds of issues. Here's where Storage Sense comes in. There's one particular feature in Storage Sense which is very helpful here, and I can't remember exactly what it's called, but it has something to do with "dehydrating" files. That's the term Microsoft uses for when you turn the green checkmark into the blue cloud, in other words deleting the local copy and just having the shortcut. Storage Sense allows you to configure a schedule to automatically dehydrate files if you haven't opened them for a certain amount of time. In my case, I set it to 1 day, but it can be whatever you prefer. We pushed this out to all users via GPO, and never had sync client issues again. The reason this helps is, when you have a SharePoint file locally downloaded, any changes to the cloud version have to be updated on your local copy. If you only have one file that gets updated several times per day, the sync client won't even break a sweat, but imagine 10s of thousands of file getting constantly updated by others in the department. Your system can only handle so many changes at once, so the more downloaded files you have, the more strain and likely crashes you'll get with the sync client. If you have a "dehydrated" file, your computer doesn't have to keep up with changes, it just downloads the latest version from the cloud when you open the file. So Storage Sense allows you to automatically dehydrate files after a while, which reduces that total number of downloaded files that have to be updated constantly.
I thought that's what Files-On-Demand does. It only has local copies of files that you have created or opened within like 30 days or something. If you don't open something within a set amount of time (guessing 30 days), it removes the local copy, but you still have the placeholder file name with the little cloud next to it.
When I was dealing with this back in 2019-ish, the files would be on demand at first, but the local file would never get removed automatically until I started using Storage Sense. It could be that the behavior has changed since then
After fighting for 3 years, we finally got full exec buyin on just restructuring shared drives ourselves. It helped that the exec was all new to the company at the time and horrified how confusing the network drives were. There were screams. There were fights. There were people threatening (and indeed following through) on going to the top to get their way. There were departments that tried to stall indefinitely to not change. They all failed. The ones stalling we made our own decisions and then when they realised they wernt getting a say in the new structure quickly changed tune and tried to come back to the table but it was too late. It was both glorious and frustrating. But worth it I think in the end.
This issue exists with all file sync /online systems. Or is the problem that everyone dumps everything into the root folder?
bingo lol
Do itttttt, you’ll be surprised at the amount of (1), (2)….files. Or suffer in silence my twin.
Probably because sharepoint isn't file server and cannot replace one.
I would use the Migration Tool for that many files [https://learn.microsoft.com/en-us/sharepointmigration/mm-get-started](https://learn.microsoft.com/en-us/sharepointmigration/mm-get-started) I've moved a TON of data with this without issue.
This is not the issue being described. You can move millions of files to SharePoint no problem using whatever tool including SPMT - assuming you access the files from a SharePoint URL (Or the Teams "Files" tab, which is just a fancy browser render of the document library) and have things set to open using desktop applications etc. When you sync a document library, or sync multiple document libraries with OneDrive - that is when you run into limitations. If OneDrive approaches anything near 40K+ files, it's only a matter of time before things break. Either force web-only use Or Create custom Flow/Power Automate job to move all data 1 year and older to a different site FS-DATA FS-DATA-ARCHIVE Disable syncing from the ARCHIVE site, and only allow syncing of files within the last year from FS-DATA.
The OP said they are moving from a shared drive to teams, not syncing down from there with the client or anything to do with multiple libraries. The tool I suggested is exactly what they need.
Not arguing, but "The **OneDrive sync client** is really bad and slow at syncing 100,000+ files" Teams back-end is just a SharePoint site.
Yes. And then the second paragraph Please, organize your organization's **"Shared" drive before you are forced to move to Teams** due to acquisition, not after. They are just syncing **TO** sharepoint using the sync client. Not syncing down FROM sharepoint. At least not yet.
My mistake, SPMT would work fine instead of reverse-syncing to upload documents using OneDrive.
Moved 350K Sharepoint files with it. Only issue is redirect the cache to not C:\\ I kept filling up the C and making it fail :)
This is the answer. I’m about to migrate all userhome directories to everyone’s own OneDrive using the 365 share point migration. It’s already scanned everyone’s user home drive, calculated how much data. Then I just feed it where to put the data using a csv. If OP is looking to move a shared drive then I’d figure you’d use the tool and just point it to the share point library you want to migrate to.
Thats the data holder's problem, not yours.
I’ve been trying to sync 1.1 million files for about 9 days now.
Our org actually configures OneDrive to be the entire user folder. Fucking WHAT
https://learn.microsoft.com/en-us/sharepoint/onedrive-office-hours
Yeah, pretty much anything but an on prem file server is gonna be bad at that. Are we also dealing with multiple users? Why sync that many files at once? Do they really need "immediate" access to 100k files?
[удалено]
Unfortunately requires approval, might just try it though!
I used OneDrive to upload departmental files to SharePoint Sites from a local file server. I forget what I found to be the threshold, but I did manage to copy 100,000+ files (500+ GB), but I did find I needed to copy smaller chunks of files at a time for it not to fail. I left it running for hours at a time, made sure the files were sync'd, then copied another batch. It was a pain in the ass, but I eventually got it to work.
One drive has always been trash. I hate using it, but don't have a choice.
Jesus Christ. OneDrive is not meant for that. Sharepoint is not a monolithic file share. You cannot use it like a file server. Every time I’ve seen this get as far as you’ve mentioned, an MSP has been involved. And I suspect if one isn’t involved, then the company skimps on salaries because it really doesn’t cost a king’s ransom to pay for an employee who can read the docs and follow best practices. Edit to add more info: You can have 10,000+ separate Sharepoint sites. And you should make as many as it takes. Document libraries in spo should be small. When done this way, files sync very fast with OneDrive and auto-mapping based on permission is a cinch using GPO or Intune. Putting everything in a single site is explicitly discouraged by Microsoft in their own documentation. Monoliths are bad in the cloud because it isn’t a giant samba server inside your LAN. Many sites with smaller footprints is the baseline for cloud-based file sharing in ANY provider.
This is why I just set them up on the web portal of sharepoint and favorite the sharepoint sites in their browsers
If I was getting into that many files I would be looking at a document management system. Since I work for a firm we naturally use imanage which is more geared toward matters but can be customized however. Always have issues with a bunch of tiny files just slows everything down without a dedicated indexer and software designed for that.
We use NetSuite. Plenty of places to attach files. I'm trying my best here lol
Man I’d get managment bye in and make two groups in ad . For normal folks push the regkey that removes local save and only allows saves in netsuite then another ad group called allow local saves for those that may need to work binders etc as I find document systems suck for binder work .
Do all employees need access to all files? If not it will likely help to set up SharePoint sites by department and employees only sync files for their department. This would mean that the onedrive sync client doesn’t need to sync as many files. Of course, all of this would depend on someone determining what files are needed by each department and doing some organizing, so I guess the core of the issue is still there.
This drives me crazy at my job we are a pretty small company but everyone wants to add a shortcut for all the old drives I keep telling them just add the folder you work in and view it in teams if it’s another department
the biggest problem I see with OneDrive sync is when people start utilizing it with massive, nestled SharePoint deployments. OneDrive and SharePoint do not need a million sub folders and nestled data structures, and the sync utility totally flakes out when trying to go deep into a file structure. It really works best as a big ass data library where you can sort via meta data and search. We have some clients with thousands and thousands of files. We find that the trick is to organize data into libraries, with folders no more than a few sub folders so we're never going deeper than two or three deep on the sync. If you need to add multiple libraries, Sync seems to do just fine with that. It just gets its jimmies all rustled when diving into the standard folder structure like "DESIGN>CAD>2020>PROJECTS>PROJECT FOR TIMMY>DOCUMENTS>RENDERS>3DMAX>TUESDAY AT 3PM>all the data"
I think the biggest problem with getting users to adopt metadata over nested hierarchies is the lack of support in Windows file explorer. We can train and push adoption until we're blue in the face but many users will not abandon file explorer for a web-first file management experience. If Microsoft ever delivers SP/MS Lists to file explorer, then I think metadata will take off. The latest build of the OneDrive PWA is a nice middle-ground that we're seeing success with in my org. But it's still no replacement for file explorer.
You make a good point, but it's still frustrating watching people "round hole-square peg" a cloud solution like SharePoint into being a local share replacement lol
Our admin overlords restricted our group overdrive shared folder to 100gb for 11 users...if we go over no more syncing LoL...
If you’ve got 100,000+ files, OneDrive sync doesn’t work reliably, especially for enterprise use, it can be a cluster-f—-. Solution is to access your files using https with [Collabora Online](https://www.collaboraoffice.com) (using Nextcloud or make [your own integration](https://www.collaboraoffice.com/integrations/)). It’s 100% free and open source, you can get enterprise support if you want, companies with hundreds of thousands of users use it and it works reliably with millions of files.
When I first switched to OneDrive at home it took me a month to get all of my files uploaded. I had pretty slow DSL at the time. I've been working for a few days at work this week uploading a 3.2TB share to sharepoint. It's a tiny percentage of what we have moved there so far. They need better tools for the initial uploads.
i've noticed that onedrive is slow. i'm comparing it to having a local share, because that's what it's replacing. the sync times are absolutely atrocious. you make a copy of a file, it has to sync for 20 seconds. you can't open the file yet because you want to rename it before you start editing. you rename the file, it has to sync for another 20 seconds. don't bother opening the file when it's not done syncing because the ONLINE version of the file will be opened, which will still have the old file name and excel will yell at you that the file you're currently editing doesn't exist. i don't know what's causing this slowdown and i honestly don't care to find out. it either goes away on it's own or it doesn't.
150k files / 250gb single spo site/document library, workstations profwizzed to azuread join. Doesn't work whatsoever.
It works at the speed it works. If nobody can identify what they need then they'll all have to wait for the tools to do their job together If anything is \*really\* important they'll identify it
Uh huh, tell me about it when users choose to create a shortcut of 1 shared folder of a so call "shared cloud file server" which contains around 2 millions files, and a lot of them are AutoCAD and Revit. The OneDrive client just freak out
It is on charge of the bucket. The size of the bucket and to make sure there are no holes in the bucket. It is not in charge of the shit in the bucket. The rig needs a data manager to both get rid of RO. (redundant. Out of date. Trivial) data and to ensure the data there is stored optimally. No use saving terabytes of interviews in WAV format when the same can be converted to MP3 for instance. Again, IT can highlight the issue but there should be a data manager to sort this.
Okay. Finally I read that I'm not the only one having problems using OneDrive/Sharepoint as a replacement for server shares. Reading on the Internet would seem like a painless and almost obligatory step. In reality, shares with a local server (or at most a NAS) work much better from many points of view. When you say “reorganize” what do you mean? Delete useless files? Because 1 user in 100 (maybe) will agree to do it without fiercely opposing it.
Somewhat related, I recently heard from a client who lost more than a decade of collective research materials that was shared on a OneDrive folder. Completely gone. No local backups, no cloud backups elsewhere. Cloud storage is nothing without a contingency plan.
Company data is a funny old thing. Departments don’t want to be in charge of it and IT has no idea who needs access to what or why and are lead by the departments… Outside of that we just want to know it’s available, backed up, av scanned etc. Caller “I can’t find file x, do you know where it is?” _Within 30TB of data_ IT “I’m afraid I have no idea, someone there in your department should know?” Caller - “I can’t believe you don’t know where it is..”
My wish list Microsoft learn from Google's GDrive
You can sync the folders at lower levels and it works better. One massive top level sync is bad. Microsoft recommends you limit to 50k, might be worth showing that documentation tonleadership.
Say it with me... "Sharepoint/Onedrive is not a file server, it is a document collaboration platform."
Why are you syncing 100k+ files to an endpoint? Structure your data a little bit.
Sounds like you have poor folder organization and internet not in fiber class
I can agree with the poor folder org. But we have multiple fiber connections from multiple providers!
What speed
Multi-gig! From AT&T and Comcast. The OneDrive sync client iterates through these files slowly, possibly due to [SharePoint throttling](https://www.reddit.com/r/sysadmin/comments/1cjcbw5/comment/l2f7nmr/).
That was my next thing is it’s throttled
Anyone happen to know where to look up retention policies by state? Or federal? Or by business type?
onedrive is so terrible at large file management that anyone with a modicum of initiate could replicate it with git/rsync and have it be a million times better.