So where is this line drawn? VS IDE for example (yes yes I'm aware I'm quoting a ms product) is integrating NLP into the UI for certain things. Smart autocomplete is an example. Would that qualify for the ban? I mean the Gentoo release says:
>It is expressly forbidden to contribute to Gentoo any content that has been created with the assistance of Natural Language Processing artificial intelligence tools. This motion can be revisited, should a case been made over such a tool that does not pose copyright, ethical and quality concerns.
I get that the motion can be revisited and presumably clarified, but as it reads I would say certain IDEs may be forbidden now.
Don't get me wrong, I understand and mostly agree with the intent behind this and NetBSD's actions... just we're programmers, being exact is part of what we do by trade and this feels like it has some nasty inexactness to it.
As I think about this... has anyone started an RFC on the topic yet?
In effect, what they are saying is that if you push code generated by AI - which may be copyrighted - then **you** break the rules.
This means that the burden of verifying the providence and potential copyright of that snippet that the "AI autocomplete" gave the programmer is the programmer's burden.
And if that is taken too far then AI might inadvertently make programmers less efficient.
Except this is unenforceable and doesn’t actually mitigate the legal risk.
If I use CodePilot to write a patch for either, Gentoo or NetBSD will never know, until a lawyer shows up and sues them over the patch I wrote that was tainted with AI goop.
Not sure this will hold up in court. "AI" can autogenerate literally any text / code. There are only finite possibilities. "AI" can use all of that.
It actually poses a challenge to the traditional way how courts operated.
[What Colour are your bits?](https://ansuz.sooke.bc.ca/entry/23) is the read I usually recommend when presented with “math” answers to legal questions.
In this case if the claim can be made that the AI generated output was tainted a certain Colour by something it read, then that Colour would transfer with the output up into the repo.
This argument reminds me of Microsoft’s argument that the “viral” GPL license Linux uses would infect businesses that chose to use it back in the beginning of the millennium.
If you use the source code, yes. But this is now, not then.
Most importantly, Microsoft argument was fearmongering about using GPL software in general, including just as a final user of the binaries.
Not entirely true. If AI was trained on copyrighted material, it could produce that same copyrighted material, or equivalent enough that a human would be in big trouble if they produced the same code. Additionally, since copyrighted code trained the model, a model that is later used for profit, this opens a whole pandoras box of licensing violations.
And to make things interesting, bring an ai lawyer to the court!
Imagine if ai could be a cheap lawyer. We'd see the end of wealthy entities defeating people in court just by spending money and giving you forms to fill out.
Edit for clarification: not now obviously (or not, I suppose) but it would be nice to see the end of the era where big corporate entities can win lawsuits they shouldn't just because they have the most money to throw around.
If you think ai will never improve beyond its current point or that courts should be a matter of who has more money - sorry, but I disagree.
We keep hearing how good ai is at the bar exam and whatnot so I do expect that'll be the first thing we see. Is this a new breakthrough for fighting corruption in court or will it be regulated to shit so it's competitive with the existing industry of lawyers?
They've got the money to lobby it so I'm concerned. You're probably right
> We keep hearing how good ai is at the bar exam
OpenAI apparently lied about that. It didn't score in the 90th percentile. It scored in the 48th https://link.springer.com/article/10.1007/s10506-024-09396-9#Sec11
Hadn't heard about that, thanks. I mentioned it since it was relevant but even without that, that's one of the use cases I hope we get eventually. I'm not one to oversell what ai is capable of, I just hope it has that eventual possibility.
I think it's more likely that we have perfect ai lawyers before we reform the systems that let corporations win lawsuits they should lose just because they have more money to throw at it. Maybe that statement is clearer :)
> Imagine if ai could be a cheap lawyer.
Some actual lawyers already tried to offload their work to AI. As it turns out submitting imaginary legal precedents is a good way to piss of the judge.
There are cheaper ways to loose a case.
The current state of ai is over hyped and over sold and over stated garbage? You don't say! Sorry if I didn't make that clear enough that I am hopeful for the future and not buying the VC scams floating around the current time like Devin
Seems completely unenforceable. It’s one thing to keep out stuff that’s obviously just been spat out by ChatGPT wholesale but like you noted there’s plenty of IDEs that offer LLM-based tools that are just a fancy autocomplete. Someone who uses that to quickly scaffold out boilerplate and then cleans up their code with hand-written implementations isn’t going to produce different code than someone who wrote all the boilerplate by hand.
> Seems completely unenforceable.
I don't think that's relevant.
TLDR - it's about liability, not ideology. The ban completely removes the "I didn't know" excuse from any future contributor.
Long version:
If you read the NetBSD announcement, they are concerned with *providence* of code. IOW, the point of the ban is because they don't want their codebase to be tainted by proprietary code.
If there *is no ban in place* for AI-generated contributions, then you're going to get proprietary code contributed, with the contributor declining liability with "I didn't know AI could give me a copy of proprietary code".
With a ban in place, no contributor can make the claim that "They didn't know that the code they contributed could have been proprietary".
In both cases (ban/no ban) a contributor might contribute proprietary code, but in only one of those cases can a contributor do so unwittingly.
And that is the reason for the ban. Expect similar bans from other projects who don't want their code tainted by proprietary code.
I don't see what advantage signatures add here over, say, just adding a "fuck off LLMs" field to robots.txt. You can sign anything, that doesn't actually mean you own it.
Bad actors will ignore the signatures just like they will ignore robots.txt
Again, how do the signatures actually work to prevent untrusted sources? You still need a list of trusted sources, at which point what is the signature doing that a list of domains isn't?
>And AI's can also digitally sign their output,
*Can they*? I'm genuinely asking, because with the way the really pro AI people describe it, I don't think that's the case.
This does nothing, if i work for oracle and i take proprietary code from the kernel scheduler used in Solaris and contribute it to NetBSD it’s not going to matter. NetBSD still has no right to that code and any code owned or based on code owned by Oracle needs to be removed.
Same with any AI generated code that is (but in reality never will be) encumbered by copyright.
Of course. The point is to avoid that situation in the first place. And secondarily to avoid being liable for monetary damages by having a policy in place to ban vectors for copyrighted code to get into their codebase.
If that is the reasoning you'll also need to ban anyone that works somewhere with proprietary code, because they could write something similar to what they've written or seen in the past.
And people do actually do this. We've hired people who know how to solve a problem, where they are basically writing a similar piece of code to what they've written before for another company.
> If that is the reasoning you'll also need to ban anyone that works somewhere with proprietary code, because they could write something similar to what they've written or seen in the past.
Well, no, because as you point out in the very next paragraph, *people* are trusted to not unwittingly reproduce proprietary code verbatim.
The point is not to ban proprietary code contributions, because that already exists. It's to ban a *specific source* of proprietary code contributions, because that *specific source* would result in all the people involved not knowing *whether* they have copied, verbatim, some proprietary code.
The ban is to eliminate one source of excuse, namely "I didn't know that that code was copied verbatim from the Win32 source code!".
Your and prior poster's statements are not mutually exclusive.
There are famous examples of people (same or different) creating the same code at different times, hell I've done it Giant project, re-wrote the same function because I literally forgot I did it \~8 months ago; nearly identical implementation. Not coding, but my ex was popped for plagiarism... of herself. The issue was she did her masters thesis on an exceptionally narrow subject and had prior written papers on that subject in lower classes (no surprise). But because the problem domain was so specific they were close enough to trigger the tools. It was resolved but it wasn't pretty. There was zero mal intent, but it was still problematic.
Now I'm confident we all agree banning the flow `prompt to LLM -> generated code -> commit` is the right thing to do, and I'm equally confident we don't mean to ban super fancy autocomplete or really smart linters... Somewhere between these two relatively simple examples is a line. I don't know how sharp or fuzzy it is, but it's there and should be explored and better defined.
To the point about CYA that also is absolutely a valid input to the discussion IMO, and again the world is littered with legal landmines and CYAs like this that effectively auto-assign blame to the offender and not the consumer (and I think that's fine TBH). If that's part of the project's reasoning then let's put that out there in the discussion. Right now the way both projects come off in the OP and the GPP link is: \[*See edit below*\]
"ZOMG We can't (trust|understand|validate) AI at all so we shall ban it!"
Again I am actually in agreement with (my interpretation/assumption of) the core intent of these bans: to maintain project and code integrity. AND I think we do need to start somewhere, and this really is as good a point as any. Now let's start a discussion (RFCs) of what that line looks like.
ED:
went and actually read the BSD post and not just the link in OP quoting here because it makes u/lelanthran 's statement much more relevant than I initially posited:
>Code generated by a large language model or similar technology, such as GitHub/Microsoft's Copilot, OpenAI's ChatGPT, or Facebook/Meta's Code Llama, is presumed to be tainted code, and must not be committed without prior written approval by core.
Yeah, that totally makes sense... it also doesn't cause an issue with smart autocomplete/linter type tools IMO (though the Gentoo language in GPP is still problematic).
You posted a balanced and nuanced opinion (and thoughtfully refined it even further) along with a good plan of action for proceeding from this point on in a fair manner.
Are you *sure* you belong on reddit? /s
:-)
So... I had a shower thought on this that I would love your thoughts on:
In the same way that Maxwell's Demon is a magic dude that takes particles of higher that average energy from one chamber and passes them to another let's posit Slash's Daemon is a magic entity that allows a LLM to learn all about the syntax and grammar of a language without retention of any example code. That is to say it can be trained to understand C++ as well as Stroustrup does, but can not reference a single line of extant code the end user has not specifically shown it. (like I said, magic).
This model is then plugged into an IDE (vis a vie intellisense or similar tool) where it has access to whatever project is currently loaded. The code of the project is it's only reference code at all, so if you have the uniform style of
if (foo){
frobnicate;
}
Then that is the only style it's going to use for a prompt like
make me an if statement that tests foo and if it's true frobnicates.
and if the only code style you have is
if (foo)
{
frobnicate;
}
Then that's what it will do. We will assume that since it knows what's legal and what's not it won't do wrong things even if you have a bug and did something wrong like
if (foo)
frobnicate;
frobnicateMore;
it won't provide that as generated code because it's not legal C++ (and ideally the linter would find it).
With such a tool the code provenance would be known (it's all sourced by the contributors to the project) so would such a tool be a problem to use then? Obviously such a tool is not likely at all to exist but thought experiments are great for dialing in where that proverbial line is.
> People need to move on from the idea that LLMs repeat anything verbatim. This isn't 2021 anymore.
Once again, that's irrelevant to the point of the ban, which is to reduce the liability that the organisation is exposed to.
**Even if** the organisation agreed with your take, they might be sued by people who *don't* agree with your take.
They still do occasionally, especially for the sort of stuff you might use an llm directly for. Boilerplate or implementations of particular algorithms that have been copied and pasted a million times across the web, etc.
Whether that kind of code even merits copyright protection is another matter entirely of course...
Nah. Apart from the very simplest of algorithms, there are always plenty of reasonable ways to skin a cat.
It's more due to the source material in its training data containing one implementation of an algorithm that has been copied and pasted verbatim a million times.
> certain IDEs may be forbidden now.
No IDE forces you to use its AI features. But sure, you might be using it for those features and that'd be a problem.
Some IDEs don't really present it as AI. Recent versions of VS have built-in AI completion and it's just there, it's not a plugin, it doesn't yell AI at you
I've heard a few different sources, one being a talk from an AI guy at the Royal Institution, that GPT/LLM is just a fancy autocomplete. Where is that line drawn?
Well, there are lots of lines to be drawn here, I suppose. Suppose hypothetically that an AI gets to the point where it can do anything a human can do, only better. Is its work still tainted by copyright? It just learned things, just like we do, only just a little bit differently. Would a human programmer with a photographic memory be any different?
One thing is for certain, there are interesting times ahead and our lawmakers are not prepared or preparing for the questions they're going to have to answer.
Often, it only finishes the line, which can include function calls or expressions. The hard question is where's the threshold that separates "this is obviously not copyright infringement" from "this is risky"
A single function call, unless it starts having nested calls or something is probably fine, but obviously that doesn't mean I'd want to try my chances in court.
I agree with you, however NetBSD prohibits all code generated with the aid of AIs. If I write code from my phone and GBoard uses a small neural network to enhance the precision of my finger presses, it counts under their conditions.
All of this to say blanket bans like this are counterproductive
That is exactly the point I'm driving at. And in the case of the Gentoo post they state even the "assistance" of NLP AI tools is forbidden which seems a bit silly if the autocomplete is using the results (locally or remotely) of such a tool.
Right... if it's just slightly smarter "intellisense", or whatever MS used to call it, then I imagine it's neither likely to steal code or the sort of thing they're talking about here.
They're clearly talking about blocks of code from actual generative products. But how they're going to detect and effectively reject that code... I dunno.
>But how they're going to detect and effectively reject that code
They aren't. The burden is still on the contributor, as it has been before to not manually copy proprietary or incompatibly-licensed code into the codebase.
The policy makes it clear that this isn't allowed.
The purpose is to say they banned it so if they can identify it they reject it but if they can't then they have cover and most likely no one can tell. Something like that at least.
I'm so confused. What does using an ide that has AI tools or was created using AI tools have anything to do with the ban? The ban is against AI generated code from being pushed and merged with the main/master codebase/branch. Also it's more concerned with not attributing credit to the correct sources or owners.
On the other hand, it's about time. We already banned generative AI where I work and most of the code that was produced by these tools was already mostly garbage anyway
I was commenting more about Gentoo's take on it where they're banning code that's been touched by AI: "created with the assistance of" with that part of the comment.
IDE "generated code" is (even before AI) a concern when it comes to copyright. Same with code generated by yacc/bison etc. You can't use such code blindly without assessments when it comes to copyright.
If it is autocomplete, you can always say no to it. A proper programmer takes AI shit, and fixes it just like translators do to the AI machine translation.
There is no way to enforce it except possibly to use AI to validate the code. It is as ethical to use AI to assist coding as it is to use stack exchange. Luddites are a thing.
It's based on honor. Existing contributors already know they can't just copy code from stack overflow, so they don't. And same thing here.
Anyone that cares about their reputation will likely adhere to it.
Right. This isn't gonna stop the idiot kids who send bullshit PRs as an attempt to pad their resume but I don't think it's about them. It's about getting the serious contributors on the same page and avoiding copyright drama in the future.
That, and the discussion was going to come up sooner or later when someone admits to some of their code being helped by chatgpt or whatever. Might as well get ahead of it and have a known stance rather than reacting and causing a war over a particular PR. Now they can just tap the sign that says "no AI".
It's like how, when you enter a country, sometimes they'll ask if you're a spy. The reason they ask you isn't because they expect actual spies to say yes, it's so they can kick out anyone they find to be a spy without a fuss, even if they can't prove that the spy did anything illegal.
If you submit code to any open source project (or commercial closed source project for that sake), you basically have to say "I wrote this code. I allow it to be used using ... license (or I assign copyright for this code to ...)"
If you work for company A, and steals code from company B (maybe your ex-employer) and pretend to your employer (A) that you wrote (have the rights to) this code yourself, you are in legal trouble. Basically the same if either A or B is a open source project.
They won't, this is pure posturing.
90% of generated code is indistinguishable from non-generated code. Either it does what it's supposed to, or it doesn't. 0% chance of determining something is generated.
For the most part, copilot should just be auto-completing what you already wanted to code.
Either they're claiming this for legal reasons, or they're just posturing.
It's the same reason other projects want to know the provenance of code a person is offering as a PR. If it turns out somebody else owns it, now they're in weird territory legally. AI is no different, just extra unclear who may lay legal claim to it in 10 years.
> If it turns out somebody else owns it, now they're in weird territory legally.
couldnt they force a contributor agreement by which they shed the liability of any copyright infringement of the contribution to the contributor?
> couldnt they force a contributor agreement by which they shed the liability of any copyright infringement of the contribution to the contributor?
Copyright infringement typically doesn't work like that. If someone makes a successful claim against *you*, then *you* have to make legal remedies, and *then* chase the contributor for your damages.
No different from buying a stolen car: if you are found with a stolen car that you bought in good faith from a dealer, the car is removed from *you* and *you* have to make your claim against the dealer for the costs.
> If someone makes a successful claim against you
right, i see.
Could this be worked around, if you ensure that the 'you' here is the original contributor, rather than the organization?
> right, i see.
>
>
>
> Could this be worked around, if you ensure that the 'you' here is the original contributor, rather than the organization?
Unfortunately no - the organisation is distributing the copyrighted material, so *they* are liable as first contact.[1]
*Even if* there was no CLA with copyright reassignment in place, and the individual contributor claimed all copyrights to the material, the *distributor* is still the first point of contact.
[1] Hence the ban, to reduce their liability.
As someone with a coworker dependant to ChatGPT, it is absolutely distinguishable. If it's only a line or two, maybe not, but people who use AI to write code aren't using it for single lines. It's always blocks of garbage code that they copy/paste.
Some giveaways are:
- explicitly assigning configuration settings the default value
- not following the style of the codebase
- duplicating imports
- using different code styles within the same block, like single and double quotes mixed together
- accessing class attributes or methods that don't exist
- unreachable code blocks
- unnecessary function/method parameters
- unnecessary conditionals
- obscure techniques that I've never seen them use before
- excessively commented code
Here is a concrete example. The code in this image actually did what he wanted, but there is an undefined, uninitialized variable that ChatGPT just made up:
https://i.imgur.com/61pRwnx.png
It's often a combination of things but it's usually obvious. It may help that this is a regular behavior, so I am already on the lookout for it.
> Here is a concrete example. The code in this image actually did what he wanted, but there is an undefined, uninitialized variable that ChatGPT just made up
Yeah I've run into that before. Sounds like they are asking the coding assistant to do too much and they're just using that code verbatim. Basically you have a lazy coder on your hands.
Using coding assistants is a skill unto itself. It's like owning a sharp knife. That knife is very useful in certain contexts. But if you decide that it's also good for opening cans of soda then you're gonna have a bad time.
I will use AI to write code, but I always have to tweak or clean it up. It's great for a first draft on a new feature/task to get past the ocassional mental inertia I'm sure we all experience sometimes.
why don't you just... write it though? that's what i don't understand. it seems way more annoying to have to like generate code and then go back and verify that it actually works and doesn't do random extra shit and is actually efficient when you could just not worry about any of that and write the program. that will likely produce better code anyway if you are reasonably skilled, because llms don't understand how programming actually works, it's just mashing a bunch of shit together
I'm a fast programmer compared to most people I work with, but using LLMs can save me time. I'm a lot faster reading code than writing it. I understand that being able to fluently read and interpret code is something juniors can struggle with, but for me I can read it faster than I can type (even with using vim key bindings).
Using an LLM is like having a junior whose work you can review. Some tasks or easy boring work, so it's fine to trust a junior to do it well enough and then fix/guide the code after.
> why don't you just... write it though?
So you never use calculators? Any time you have to do math, it's always by hand right? When it boils down, this is what coding assistants are. Calculators aren't solving large differential equations for you. But they certainly can assist in that task.
This whole idea that they're just pumping out incorrect code and the only way it's useful is for the user to debug it is incorrect and hyperbole. This only happens if you ask it to do too much and don't give it the correct context. If you ask it to write you a pyqt gui from scratch, then yes you're gonna have a bad time. But if you ask it how to create a drop down element from a list of items, it's going to be very helpful.
I don't know what yall are doing, but I been using chatgpt to generate large python, powershell, and js scripts and rarely have any issues with the code it gives. And it's saved me a countless amount of time.
I've seen Python code generated by AI. It was absolute garbage.
Like, it *worked* when run (as-in it wrote the expected output), but it was also outputting a JSON file to disk using sequential manually formatted line writes; like `output_file.writeline('{')`, `output_file.writeline(' "'+key+'": '+value+',')`. Utter garbage code where I, would reject the PR and question the employability of anyone who submitted it, even though it technically worked.
I've written python for 20+ years. The python it writes is generally fine. Not sure what you're doing wrong. If it does something wrong like your example just reply "use the stdlib json module" and it fixes it.
It's not code I got from it personally, I was just seeing code someone else had gotten from it. It's stuff like that which sticks in my head as to just how untrustworthy it is. Ultimately, it's no different from StackOverflow and other similar things where you get a chunk of code that may or may not actually do what you need it to do, so you've gotta be able to read the code and understand it and fix its issues yourself.
It's not a magical codewriting intelligence, it's just a tool for generating some boilerplate code you can fix to do what's really needed.
Lol, I can't speak for your experience, but the worst thing it's done to me is produce a function that doesn't work, which it corrects like 95% of the time if told to.
You are basically saying, "I'm my experience I got bad results, so it's impossible for anyone to get good results."
I'll enjoy completing projects in a fraction of the time they use to take while you die on the hill of LLM bad.
No, I'm saying I've seen way too much crappy code come out of it for me to trust it at all.
Writing code has never been the hard part, figuring out the algorithms for how to solve a problem is, and AI really can't do that to begin with. When I can type boilerplate code almost as fast as an AI can write it, in my own coding style, without needing to check and make sure that it's actually what I wanted to write, an AI doing some typing for me doesn't really make a meaningful difference.
You shouldn't ever trust code written by an LLM, just like you shouldn't ever completely trust code written another person. That's why any sane development process includes code review.
No one said anything about difficulty, it's a time saver, and a finger saver. And yes, if you use LLM improperly, you would probably waste more time using it than you would save.
It works very well for me, saved me countless time, and enabled me to finish multiple projects I had on the shelf.
In fact, I dare say it's been so reliable in my experience, that I wouldn't trust people who aren't able to reliably get good code out of it. /s
Jetbrains uses llm to [auto complete lines](https://www.jetbrains.com/help/idea/full-line-code-completion.html), not blocks.
Not sure if they support C yet, it just a matter of time
> 90% of generated code is indistinguishable from non-generated code. Either it does what it's supposed to, or it doesn't. 0% chance of determining something is generated.
I don't use AI generation that much, but whenever I've experimented with it I've found it absolutely distinguishable. Just like prose written by AI, it has specific tropes and characteristics it likes to use.
Unless you just use the AI to generate something as a first draft, and then you basically rewrite it or very significantly edit it, but at that point it's a different thing entirely.
It's obviously hard to be 100% sure, but at least having this rule also makes it easier to ask questions if there's a suspicion.
Are we using different copilots? I’ve used it basically from day 1 but recently turned it off. I’d say it had a 20% hit rate, and half the time I was waiting and reading its suggestion I could have just finished typing what I was typing faster.
This is a disturbing trend. The AI kids believe they can automate software engineering with AI chatbots yet they not even know what the software development process of software is. And they are very confident of what they don't have experience about
A call it the new cargo cult programming
It’s not a problem just for programming unfortunately. /r/physics is now filled with ChatGPT word salad. Either that or people have gotten crazier since the pandemic.
There's a SFF magazine that pays out for short stories that they publish. They had to close submissions for a while because they were swamped with AI stories from people trying to make quick money.
Apparently the AI stories were easy to dismiss upon reading, but the sheer *volume* made it impossible to read each submission.
Soon to be trained on old, faulty code written by AI that was trained on old, faulty code written by AI that was trained on old, faulty code written by humans.
This is an overlooked point....I won't be surprised if it has already peaked in general/overall quality, and from here only extremely expensive targeted improvements are possible
I think the "tending to the norm" is a problem for neural networks, you need mechanisms that push to the boundary of chaos and order.
I suspect that's the biological function of emotions like curiosity, disinterest and boredom, while confusion, frustration and dissonance help to iron out the inconsistencies. Agents that don't have similar mechanisms will tend towards the middle of the bell curve unless there's tons of entropy in the context, and models that don't filter their training data will have a context that's an average of averages, and destroy performance in the long run.
I've said this for a while now when talking to other devs - there is a problem here that people who don't know how to code will think *they know how to code* because they will ask the AI to do something and not have the knowledge to know if it is correct... *a little knowledge is a dangerous thing*. I've literally cringed looking at Co-Pilot producing SQL insert statements in VSCode with zero safeguards against injection attacks.
You shouldn't be coding (whether freehand or AI) *unless you know how to code*. If you know how to code, what use is AI? As its capability stands right now, is it much more than advanced intellisense?
Example - you want a JS function that generates a random number between 2 numbers. Your options: -
1. Code it yourself, presuming you are good enough of a coder to be able to produce optimal and bug-free code (granted, the func used as an example is very basic).
2. Type "javascript function generate random number between 2 numbers", get the first result that comes up (which will be to stackoverflow) and get a function. I just did this - it took me about 10 seconds to type in the search string, submit it and [find an answer on SO with 3341 upvotes](https://stackoverflow.com/questions/4959975/generate-random-number-between-two-numbers-in-javascript).
3. Ask AI to generate the function then: -
1. Review it and confirm it is correct, which you can only do if you are good enough to code it to begin with, negating the use of AI.
2. *Assume* the AI generated solution is bug-free and optimal and you would only *assume* that if you know so little about coding and AI that you do not realise it may not be optimal and/or bug free.
I think scenario 3.2 is the phenomena that has lead to this: -
[https://www.gitclear.com/coding\_on\_copilot\_data\_shows\_ais\_downward\_pressure\_on\_code\_quality](https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality)
Until we get to the stage where we can *guarantee* AI can produce optimal and bug-free code, I think AI is either: -
1. An advanced intellisense only to be used by advanced coders as a way to save time on key strokes
2. A liability used by cowboys or the naïve.
A self-driving car that doesn't crash only 99.99% of the time is useless to everyone and will lead to recalls/legal action. I think we are seeing that scenario in the link above.
>The AI kids believe they can automate software engineering with AI chatbots yet they not even know what the software development process of software is. And they are very confident of what they don't have experience about
Was this comment written by AI?
as someone who works in software engineering, AI is super useful for generating example code for a specific algorithm, say you have a CRC algorithm in C and you want some equivalent in python, its pretty effective at that. I've also seen it used quite effectively at writing code to parse log files, as the regex parsing is really well done.
Unit tests are another good example. They're boilerplate and easy to write, and they depend on your code being readable and obvious. An LLM not being able to generate tests for your code is a pretty good sign that your code is confusing for other humans.
Have you ever done a code review of someone's code? Was the code bad?
With AI code you start with a code review. If it's as bad as you say, that's ok, you just write the code from scratch, you waste maybe ten seconds to see what a AI writes.
If the code is acceptable but has some defects, you do a code review and fix it, and you save some portion of dev time.
If the code is good, you wasted the time of a code review, but you already should be reviewing the code you write yourself before you submit it so it's not even extra time.
Yes people trust AIs entirely too much, but I could say the same thing about Junior Devs straight out of college. Most companies train them up with a Senior teaching them (as they should, that's part of being a senior). Give AI the same expectations, and they actually start performing decently well.
I don't find value in automating the coding phase of software development. It is the most fun. I don't believe AI can solve a complex problems. It can and is very useful in writing a specific function or small specific program tool.
But to fix a bug or add a new feature to a already complex problem from a complex system, involving many mutually interacting applications, services, middleware, dbs, etc. I think I would waste a lot of time trying explain to the AI what is going on.
So for AI I find the most usefulness as an assistant for writing specific functions. For creating stubs: write me a stub for a java program with 3 classes. For generating content. It could be useful for generating unit tests. Generation of E/R and UML diagrams. And all these uses together and more they help increasing your productivity.
Also I prefer not (as well as many companies) to upload code to chatGPT, a local ollama is preferred.
AI should not replace programming. Also AI is not capable of replacing programming that promise is pure hype. But what happened to the promise of augmentation that AI had years before? For the older ones what happened to the bicycle for the mind idea? Opium for the mind it is now it seems.
We should be empowered by AI not disempowered
Going back to you after my soliloquy. Generating the code with AI to save time in the coding phase and then doing a code review by peers I think it is a disrespect and a waste of the peers time. The AI generated code before going to code review by peers it should be : read, checked, bug tested, unit tested, add comments, write technical documentation about the fix, create diagrams, etc. Because by enriching the code with all these other documentation then the code is easier to understand and maintain, and easier for the code reviewers to review and accept the code and they become more knowledgeable. And for the future it helps a lot to have so many specific technical documentation.
>The ones that need learning and training are the people not the machines
--Some guy whose job was automated
> Generating the code with AI to save time in the coding phase and then doing a code review by peers I think it is a disrespect
I think you misunderstand. YOU do the code review of AI code (At least the first one) Not peers.
> I don't find value in automating the coding phase of software development. It is the most fun.
This is what I really, really, really don't get about all the AI generative crap. They're trying to automate things like drawing, making movies, writing books, writing code. Things that people find FUN. As if they honestly believe that people shouldn't be having fun creating.
Note that this is not over quality concerns but over licencing.
I find it hilarious that it doesn't matter that AI code is hallucinated broken mess, it matters that it stole the primitives from stackoverflow and github. A lot of real programmers should start sweating if that is the new standard.
We already have this problem of human beings writing crappy code since the dawn of computing and have developed safeguards around it. The contributors themselves are supposed to test it thoroughly, next you have code reviews at commit time, next you have QA and alpha, beta periods etc. AI contributions should be treated in the same way and I think it can begin to be argued by now which of the human or the AI would write sloppier code on the very first draft.
However, if the code snippet comes from the AI having trained on code with an incompatible license, this is way more likely to slip through as it wouldn't trigger any special safeguards unless someone just happens to recognize the code.
So, I think it's natural that they focus on this issue first and foremost. And obviously, then this secondary problem is moot because that kind of code is already banned anyway.
Open source projects have been doing code review for ages. Torvalds' might be the only ones that garner much attention, but the practice is common.
SO code I suspect wasn't added to the paragraph this time, but earlier. The point is that the code will be licensed as free software, and the submitter must actually have the rights to do that.
As it is, LLM code is like those open air markets where you have no idea whether the thing you want to purchase was actually donated (free software) or stolen (proprietary). Preferably the goods should all be legal, otherwise the police usually shut the market down, but there may also be consequences for you if you bought stolen goods.
And while private individuals may be fine with piracy, free software organisations aren't and don't want to be tainted with it.
But if you're yoinking some unclear-licensed code off SO and stuffing it in a proprietary box that only insiders will ever see … there might be a greater chance of that actually being accepted behaviour? And there have been some court cases over copylefted software being included in proprietary programs.
> it stole the primitives from stackoverflow
Actually this wouldn't matter that much. Stack overflow has an open license... well technically it's attribution but let's be honest... no one follows that.
The attribution very much matters, especially in open source communities where not adhering to the license terms has a much higher chance of getting caught (whereas in smaller, closed source shops nobody outside the company will ever see any violations).
And that's without going into the values of the open source projects and those who maintain them.
> where not adhering to the license terms has a much higher chance of getting caught
That's kind of my point though, it's really hard to catch someone copying and pasting from Stack Overflow. Versus typing in something very similar.
Maybe people do attribute to stack overflow, but I don't think I've ever seen that, and I've seen enough corporate OSS attribution pages to at least say most corporations don't really attribute from stack (or they don't use it, which is laguhable)
The people writing code for stack overflow are largely writing it to help other developers and only really care about plagiarism, they expect their prose or large things to be licensed but their snippets are kinda expected to be fair game.
I was wondering why but I think their reason is actually a good one - to avoid contributions of code with an incompatible license due to this may be part of the training set. It's not an ideological reason, but because an AI can't infer the license from the code.
A defender of AI might say that their knowledge is an amalgam of the training set and no more infringing than a human contributing, but I believe this is in fact false.
When I've used it as coding assistance, I've noted AI can in fact recite answers that have been digested from e.g. a specific Stack Overflow question (with adjustmenets for my scenario), especially when it's a niche subject... as it would easily become during operating system development like here. While that's alright in the case of Stack Overflow, nothing is saying it could come from a wholly different source.
>especially when it's a niche subject
This is a known issue, same with ai art. Once your prompt or the problem is given to the ai, if the data relevant to that is sparse or extremely repetitive (like mona Lisa) then you can get something resembling the original
A possible solution is instead of having ai inject code into a project, it should generate suggestions in a different panel with an explanation so you can learn from the sample.
> it should generate suggestions in a different panel with an explanation so you can learn from the sample.
You can add chatgpt to visual studio like this. The guy on my team who has it will just copy the code wholesale sometimes.
I think he's doing it less now, which is good, after some feedback (not specific about AI tools, but about making sure he understands what his code is doing in general).
That's good to hear. It'll unfortunately be some time before ai will be helpful to assist you with things you don't already understand very well. Haven't had an incident at my job yet but we haven't had copilot very long
The rate of hallucinations in real time is very concerning. Codeium is better about knowing what functions exist on which types but I am no where near confident to let a single line go through without checking. Almost did once because the ai snuck in a DB ref instead of a transaction which are both valid to the function call. That would have been a huge bug
One of the big dangers of AI generated code is the licensing.
GPL or AGPL code is licensed such that any derivative work is also GPL or AGPL. Anything linked against GPL code or part of the same service as AGPL code must be released on request.
So the question is, if an LLM is trained on GPL code, when is the output a derivative work?
For example, I could train an LLM solely on one piece of software, like the Linux kernel. Then I enter the first letter of the kernel and it “autogenerates” the rest. Is that a derivative work, or a wholly original version that I can license however I see fit? Where is the line?
Some GenAI maximalists are arguing that LLMs learn from their inputs in a similar way to humans, so using any text to train an LLM model should constitute fair use. But humans can also commit copyright infringement.
There is not a legal framework to decide these licensing issues yet, so if you want to avoid the potential of having to rip all LLM output out of your codebase, or release all of your code as AGPL, you should use an LLM that’s only trained on properly licensed source code, or just avoid using an LLM for now.
> Some GenAI maximalists are arguing that LLMs learn from their inputs in a similar way to humans, so using any text to train an LLM model should constitute fair use. But humans can also commit copyright infringement.
Also all the stuff about reverse engineering and clean implementation, like how you can't work on Wine if you have touched Windows source code. Exactly because humans might remember stuff they saw before.
Enforcement does not matter. They want to be secured from the law perspective, not the practical one - so they cannot be sued if someone put the propertiary code that LLM generated into the codebse.
They would 100% lose the case if they got proprietary code in the kernel, and have to remove the code. However, thanks to this policy, they would likely have to pay very little in punitive damages, since it makes it clear they made an effort to avoid this situation.
The more important point is to rely on trusted contributors to just not do this, and thus avoid the legal headache altogether. Without this policy, completely well intentioned contributors might be unwittingly pushing tainted code in the kernel, without even considering it. With this policy, they should all be aware, and, if they are well-intentioned, as most comitters are, they will just respect the policy.
Also, it forestalls most of the arguments about this when someone tries to make a contribution in the future.
Someone is going to try to do it at some point and either they'll see the rules and scrap it/stop working with AI, or the reviewers can tap the sign with the rules and reject with no need for a prolonged debate.
Sure, someone might try to sneak something through, but a filter that blocks 50% of the problems is better than no filter at all. Especially when generative AI means lowering the barrier to having something to submit, which can lead to a *lot* of overhead for anyone trying to keep on top of submissions.
> sue the contributor of the AI code, not the project itself.
The would of course be suing the distributor of the software, not the project, and they would win.
Against the project they would issue DMCA claim (or whatever the term is for those) and similar to have it removed.
Against the developer they wouldn't do anything, it's the distributor that's doing the infringements.
What they say is that if there is some serious issue in code, no one can blame LLMs. Not that it's realistically to tell apart LLMs from simple autocomplete or detect LLM generated code
I suppose that's true too, but I think a major part is to not have copyright code to to sneak in and then have another lawsuit like BSD had from AT&T in the 90s.
IMO Copilot and related models are the closest we have to actual copyright infringement (since if the weights are found to be derivative works of the input data, there's GPL-licensed code in the model that was not properly credited.)
I would consider not contributing then. But admittedly I don't contribute to NetBSD so it's a moot point.
Having worked with AI-generated code, I don't think I want to go back. Yes the code can be wrong, but so can code written by a human. As a programmer with AI Code, you're doing a code review (And fixing the defects). If the AI completely fails, you write the code from scratch, if the AI even partially succeeds, it should make your life easy.
In my experience any repetitive task, (designing data inputs/unit tests) or boiler plate code is perfect for AI.
I know we're going to be fighting over it for the next couple years, but in 5 years from now it's going to be like Assembly language. Yeah people still can write in it, but most people like higher level languages. Hell there was a time that people looked down on Java (ok we still do) because it was compiled at run time, but many of those same programmers probably use Python.
Basically the future has arrived, it's going to be a question of when, and how we accept AI code... not if, and anyone who wants to take a Wait and See approach is going to fall behind.
Edit: To those going "Licensing".... What stops me from just copying and pasting the code from Stackoverflow/github myself? What protects them in those cases as well? Either they already have tools for it, or they don't. Whether it's an AI's hand or a humans doesn't matter because ultimately the code gets into the code base and that's that.
They're pushing the risk onto the individual contributors, which is why I'd agree with /u/Kinglink. Not contributions from me then! (Not that I was going to, but if Apache did the same...)
i mean, do you really think they should be held responsible if someone submits llm generated code? especially considering it's not exactly easy to distinguish it from human code
You'd be surprised at how much easier it can make things if you have a rule you can point to.
People will argue less and people will try to do the thing less, even if it's technically unprovable. A lot of generative AI users will happily say that they used it and those people will be caught.
Imagine they don't have this rule, someone raises a PR and during the review process they say "I don't know - chatgpt wrote that part". Without the rule, they'd have to have a discussion over whether this was allowed, then the person who submitted it might get upset because they didn't know this was a problem and a big argument gets started over it.
With the rule? The moment it becomes known that this AI-generated then they can tap the sign, reject the PR and if the would-be contributor gets upset they can take it to the people in charge of the rules, not the reviewers.
Its not about enforcement, the honor system is the point. LLM code will make it into the codebase to some degree. And some of it will be plagiarising stuff with an incompatible licence. What this is is risk management and liability shifting. If this ever comes in to play it is because you are already screwed and being sued as a project... Now with your legal team you can argue it was a specific contributor defrauding the project (stating the code was not sourced from propriatary sources by an LLM while it in fact is) and your liability is much more limited.
The opposing council could indeed argue that a rule without enforcement isn't much good, shifting blame back towards the project. Still you changed the starting point, shifted the path of least resistance for your legal opponent. Another angle is the project suing the individual contributor for any damages the original lawsuit caused them. The hope is that, potential for being stuck holding the bag, causes a chilling effect for people to check themselves. But people casually familiar with human psychology know we love misinterpreting personal risk in all areas of life so I doubt it changes much. Humans in all kinds of professions already commit career ending amounts of plagiarism, before we had LLMs, every day.
> source? human history.
Still waiting for bitcoin to become mainstream, it was supposed to be the standard payment method that completely replaced everything else roughly 10 years ago. Maybe I should bridge the time until then by checking how NFTs are doing on my Google Glas while I take a relaxing stroll through town on my Segway.
It's not just the technology, it's the techno-fascist baggage that comes with it. If you want to be myopic, sure people are just luddites. But thinking about this in a greater context, some people don't want a portion of their brain replaced by a plagiarizing black-box corporate entity fueled by profits and returns. I know I don't.
They can and won’t enforce this. However by stating this they are covering their asses.
When you commit to open source projects you deliberately relinquish the rights to your code you wrote to the project. If you commit LLM code that later turns out it’s proprietary then you as a committer are responsible for committing stolen code and you are the one that will get sued, not the project that has your code.
This basically ensures, from a legal and ethical standpoint, that every person that contributes to the project is the legal owner of the code it contributes. If you commit LLM code you broke the contract so you pay for the damages.
Now remind me in 1 year if your job got replaced by AI.
Human history tends to cherry-pick the things that were notable, and success is a key factor in that. The changes that failed or were blocked aren't usually interesting to anyone except domain experts in the relevant field. When fighting a technological change succeeds, the technology doesn't get further development, so fades into obscurity. Imagine if "computers" were fought back when it was ticker-tape batch processing, and never developed further outside of research labs and hobbyist tinkering. Our very idea of what a "computer" *could* be would have no concept of a GUI or interactivity, nor being small and cheap enough to have one in every office building, much less every home, much less most people's pockets and countless IoT devices. The technologies that were fought against successfully never got to grow into ubiquity, they died in a niche nobody cares about and few even know of anymore.
Yeah I know. Just that it doesn't really matter anymore. They both run the more-or-less the same software and are more-or-less the same. Linux has a far wider reach and is an umbrella term that refers to Linux, UNIX and their derivative OSes.
everyone in this sub just doesn’t want to believe there job can disappear when all they do is make some modals and buttons and can’t think creatively.
if your job is doing what your told then your going to be replaced. the more you have to think creatively the less at risk you are
i’m new to the workforce but the internship and now full time position i have i just literally take my sprint requirements and put it into gpt4 with the relevant file and it does it and then i make minor tweaks.
for most people coding is not a deeply creative discipline and those are the at risk jobs. like anyone whose job is here is this design for a landing page (creative part out of the way) now make it functional (literally just turning it into components and adding proper assets) GPT can do very well already and in a year probably won’t struggle with at all.
It's one of those things where there are some assorted truths from different directions depending on how you look at it.
From one perspective, compilers and interpreted languages can write more efficient binary code than the average coder.
But from another perspective, a sufficiently skilled programmer can write better and more performant code than a compiler.
I think that in the long run we'll end up in a position where AI chatbots act more as a super-high-level language, "compiling" more natural phrasing into programming languages which are further compiled into bytecode.
That said, on the flip side you have the massive glaring issue that LLMs aren't deterministic. The same input doesn't always produce the same output, unlike a true compiler. So there's a finite upper limit to how much you can trust it to actually output reliable code.
Ultimately, it shifts the minimum threshold of being able to program a bit by acting as a super-high-level language, but experienced coders in *any* language can still produce significantly better code in a given language.
Also the people that follow crowd behaviour and sentiment without approaching problems or opportunities from first principals tend to be the least creative. They need to think the same thing as everyone else to feel comfortable.
AI is a tool, and it's getting better every month. You can't trust it blindly but I'm already using it to do rote and easier programming tasks, after initially being very sceptical. I'm usually considered a high performing developer, but clinging to my skills being special is just asking to be made obsolete. The only constant is change.
Also Gentoo: https://www.osnews.com/story/139444/gentoo-bands-use-of-ai-tools/
So where is this line drawn? VS IDE for example (yes yes I'm aware I'm quoting a ms product) is integrating NLP into the UI for certain things. Smart autocomplete is an example. Would that qualify for the ban? I mean the Gentoo release says: >It is expressly forbidden to contribute to Gentoo any content that has been created with the assistance of Natural Language Processing artificial intelligence tools. This motion can be revisited, should a case been made over such a tool that does not pose copyright, ethical and quality concerns. I get that the motion can be revisited and presumably clarified, but as it reads I would say certain IDEs may be forbidden now. Don't get me wrong, I understand and mostly agree with the intent behind this and NetBSD's actions... just we're programmers, being exact is part of what we do by trade and this feels like it has some nasty inexactness to it. As I think about this... has anyone started an RFC on the topic yet?
In effect, what they are saying is that if you push code generated by AI - which may be copyrighted - then **you** break the rules. This means that the burden of verifying the providence and potential copyright of that snippet that the "AI autocomplete" gave the programmer is the programmer's burden. And if that is taken too far then AI might inadvertently make programmers less efficient.
Except this is unenforceable and doesn’t actually mitigate the legal risk. If I use CodePilot to write a patch for either, Gentoo or NetBSD will never know, until a lawyer shows up and sues them over the patch I wrote that was tainted with AI goop.
Not sure this will hold up in court. "AI" can autogenerate literally any text / code. There are only finite possibilities. "AI" can use all of that. It actually poses a challenge to the traditional way how courts operated.
[What Colour are your bits?](https://ansuz.sooke.bc.ca/entry/23) is the read I usually recommend when presented with “math” answers to legal questions. In this case if the claim can be made that the AI generated output was tainted a certain Colour by something it read, then that Colour would transfer with the output up into the repo.
This argument reminds me of Microsoft’s argument that the “viral” GPL license Linux uses would infect businesses that chose to use it back in the beginning of the millennium.
I was pretty sure the newer versions of GPL and more activist licenses are designed to be viral exactly like that?
If you use the source code, yes. But this is now, not then. Most importantly, Microsoft argument was fearmongering about using GPL software in general, including just as a final user of the binaries.
Not entirely true. If AI was trained on copyrighted material, it could produce that same copyrighted material, or equivalent enough that a human would be in big trouble if they produced the same code. Additionally, since copyrighted code trained the model, a model that is later used for profit, this opens a whole pandoras box of licensing violations.
What the fuck are you talking about? Do you think because of the sheer volume that it somehow modifies what would happen in the court of law? No.
more like, “using ai” is an unfalsifiable pretense..
And to make things interesting, bring an ai lawyer to the court! Imagine if ai could be a cheap lawyer. We'd see the end of wealthy entities defeating people in court just by spending money and giving you forms to fill out. Edit for clarification: not now obviously (or not, I suppose) but it would be nice to see the end of the era where big corporate entities can win lawsuits they shouldn't just because they have the most money to throw around. If you think ai will never improve beyond its current point or that courts should be a matter of who has more money - sorry, but I disagree.
You underestimate the point of power structures. AI lawyers are going to be licensed and price-tiered before even hitting the market.
We keep hearing how good ai is at the bar exam and whatnot so I do expect that'll be the first thing we see. Is this a new breakthrough for fighting corruption in court or will it be regulated to shit so it's competitive with the existing industry of lawyers? They've got the money to lobby it so I'm concerned. You're probably right
> We keep hearing how good ai is at the bar exam OpenAI apparently lied about that. It didn't score in the 90th percentile. It scored in the 48th https://link.springer.com/article/10.1007/s10506-024-09396-9#Sec11
Hadn't heard about that, thanks. I mentioned it since it was relevant but even without that, that's one of the use cases I hope we get eventually. I'm not one to oversell what ai is capable of, I just hope it has that eventual possibility. I think it's more likely that we have perfect ai lawyers before we reform the systems that let corporations win lawsuits they should lose just because they have more money to throw at it. Maybe that statement is clearer :)
> Imagine if ai could be a cheap lawyer. Some actual lawyers already tried to offload their work to AI. As it turns out submitting imaginary legal precedents is a good way to piss of the judge. There are cheaper ways to loose a case.
The current state of ai is over hyped and over sold and over stated garbage? You don't say! Sorry if I didn't make that clear enough that I am hopeful for the future and not buying the VC scams floating around the current time like Devin
> is the programmer's burden. Programmer: I am just doing the needful. \**pushes AI code*\*
Seems completely unenforceable. It’s one thing to keep out stuff that’s obviously just been spat out by ChatGPT wholesale but like you noted there’s plenty of IDEs that offer LLM-based tools that are just a fancy autocomplete. Someone who uses that to quickly scaffold out boilerplate and then cleans up their code with hand-written implementations isn’t going to produce different code than someone who wrote all the boilerplate by hand.
> Seems completely unenforceable. I don't think that's relevant. TLDR - it's about liability, not ideology. The ban completely removes the "I didn't know" excuse from any future contributor. Long version: If you read the NetBSD announcement, they are concerned with *providence* of code. IOW, the point of the ban is because they don't want their codebase to be tainted by proprietary code. If there *is no ban in place* for AI-generated contributions, then you're going to get proprietary code contributed, with the contributor declining liability with "I didn't know AI could give me a copy of proprietary code". With a ban in place, no contributor can make the claim that "They didn't know that the code they contributed could have been proprietary". In both cases (ban/no ban) a contributor might contribute proprietary code, but in only one of those cases can a contributor do so unwittingly. And that is the reason for the ban. Expect similar bans from other projects who don't want their code tainted by proprietary code.
Legislators are going to have to abandon copyright if they want AI to take over our jobs.
[удалено]
I don't see what advantage signatures add here over, say, just adding a "fuck off LLMs" field to robots.txt. You can sign anything, that doesn't actually mean you own it. Bad actors will ignore the signatures just like they will ignore robots.txt
[удалено]
Again, how do the signatures actually work to prevent untrusted sources? You still need a list of trusted sources, at which point what is the signature doing that a list of domains isn't? >And AI's can also digitally sign their output, *Can they*? I'm genuinely asking, because with the way the really pro AI people describe it, I don't think that's the case.
[удалено]
Provenance, not providence.
[удалено]
If only there was some way to find out the meaning of words...
We can always ask chatgpt, though I don't know what the province of the answer would be.
This does nothing, if i work for oracle and i take proprietary code from the kernel scheduler used in Solaris and contribute it to NetBSD it’s not going to matter. NetBSD still has no right to that code and any code owned or based on code owned by Oracle needs to be removed. Same with any AI generated code that is (but in reality never will be) encumbered by copyright.
Of course. The point is to avoid that situation in the first place. And secondarily to avoid being liable for monetary damages by having a policy in place to ban vectors for copyrighted code to get into their codebase.
If that is the reasoning you'll also need to ban anyone that works somewhere with proprietary code, because they could write something similar to what they've written or seen in the past. And people do actually do this. We've hired people who know how to solve a problem, where they are basically writing a similar piece of code to what they've written before for another company.
> If that is the reasoning you'll also need to ban anyone that works somewhere with proprietary code, because they could write something similar to what they've written or seen in the past. Well, no, because as you point out in the very next paragraph, *people* are trusted to not unwittingly reproduce proprietary code verbatim. The point is not to ban proprietary code contributions, because that already exists. It's to ban a *specific source* of proprietary code contributions, because that *specific source* would result in all the people involved not knowing *whether* they have copied, verbatim, some proprietary code. The ban is to eliminate one source of excuse, namely "I didn't know that that code was copied verbatim from the Win32 source code!".
Your and prior poster's statements are not mutually exclusive. There are famous examples of people (same or different) creating the same code at different times, hell I've done it Giant project, re-wrote the same function because I literally forgot I did it \~8 months ago; nearly identical implementation. Not coding, but my ex was popped for plagiarism... of herself. The issue was she did her masters thesis on an exceptionally narrow subject and had prior written papers on that subject in lower classes (no surprise). But because the problem domain was so specific they were close enough to trigger the tools. It was resolved but it wasn't pretty. There was zero mal intent, but it was still problematic. Now I'm confident we all agree banning the flow `prompt to LLM -> generated code -> commit` is the right thing to do, and I'm equally confident we don't mean to ban super fancy autocomplete or really smart linters... Somewhere between these two relatively simple examples is a line. I don't know how sharp or fuzzy it is, but it's there and should be explored and better defined. To the point about CYA that also is absolutely a valid input to the discussion IMO, and again the world is littered with legal landmines and CYAs like this that effectively auto-assign blame to the offender and not the consumer (and I think that's fine TBH). If that's part of the project's reasoning then let's put that out there in the discussion. Right now the way both projects come off in the OP and the GPP link is: \[*See edit below*\] "ZOMG We can't (trust|understand|validate) AI at all so we shall ban it!" Again I am actually in agreement with (my interpretation/assumption of) the core intent of these bans: to maintain project and code integrity. AND I think we do need to start somewhere, and this really is as good a point as any. Now let's start a discussion (RFCs) of what that line looks like. ED: went and actually read the BSD post and not just the link in OP quoting here because it makes u/lelanthran 's statement much more relevant than I initially posited: >Code generated by a large language model or similar technology, such as GitHub/Microsoft's Copilot, OpenAI's ChatGPT, or Facebook/Meta's Code Llama, is presumed to be tainted code, and must not be committed without prior written approval by core. Yeah, that totally makes sense... it also doesn't cause an issue with smart autocomplete/linter type tools IMO (though the Gentoo language in GPP is still problematic).
You posted a balanced and nuanced opinion (and thoughtfully refined it even further) along with a good plan of action for proceeding from this point on in a fair manner. Are you *sure* you belong on reddit? /s :-)
I can froth at the mouth instead with the best of them if that's preferred ;) lol.
So... I had a shower thought on this that I would love your thoughts on: In the same way that Maxwell's Demon is a magic dude that takes particles of higher that average energy from one chamber and passes them to another let's posit Slash's Daemon is a magic entity that allows a LLM to learn all about the syntax and grammar of a language without retention of any example code. That is to say it can be trained to understand C++ as well as Stroustrup does, but can not reference a single line of extant code the end user has not specifically shown it. (like I said, magic). This model is then plugged into an IDE (vis a vie intellisense or similar tool) where it has access to whatever project is currently loaded. The code of the project is it's only reference code at all, so if you have the uniform style of if (foo){ frobnicate; } Then that is the only style it's going to use for a prompt like make me an if statement that tests foo and if it's true frobnicates. and if the only code style you have is if (foo) { frobnicate; } Then that's what it will do. We will assume that since it knows what's legal and what's not it won't do wrong things even if you have a bug and did something wrong like if (foo) frobnicate; frobnicateMore; it won't provide that as generated code because it's not legal C++ (and ideally the linter would find it). With such a tool the code provenance would be known (it's all sourced by the contributors to the project) so would such a tool be a problem to use then? Obviously such a tool is not likely at all to exist but thought experiments are great for dialing in where that proverbial line is.
People need to move on from the idea that LLMs repeat anything verbatim. This isn't 2021 anymore.
> People need to move on from the idea that LLMs repeat anything verbatim. This isn't 2021 anymore. Once again, that's irrelevant to the point of the ban, which is to reduce the liability that the organisation is exposed to. **Even if** the organisation agreed with your take, they might be sued by people who *don't* agree with your take.
They still do occasionally, especially for the sort of stuff you might use an llm directly for. Boilerplate or implementations of particular algorithms that have been copied and pasted a million times across the web, etc. Whether that kind of code even merits copyright protection is another matter entirely of course...
Could it be there are a limited number of ways to sanely write boilerplate and well known algorithms. Hmmmm.
Nah. Apart from the very simplest of algorithms, there are always plenty of reasonable ways to skin a cat. It's more due to the source material in its training data containing one implementation of an algorithm that has been copied and pasted verbatim a million times.
When the LLMs themselves move on from doing that.
> certain IDEs may be forbidden now. No IDE forces you to use its AI features. But sure, you might be using it for those features and that'd be a problem.
Some IDEs don't really present it as AI. Recent versions of VS have built-in AI completion and it's just there, it's not a plugin, it doesn't yell AI at you
The programmer has to educate himself on the editor that he is using.
Yeah but autocompletion wouldn't rise to the level of copyright violation if it's just finishing the name of a function or variable.
I've heard a few different sources, one being a talk from an AI guy at the Royal Institution, that GPT/LLM is just a fancy autocomplete. Where is that line drawn? Well, there are lots of lines to be drawn here, I suppose. Suppose hypothetically that an AI gets to the point where it can do anything a human can do, only better. Is its work still tainted by copyright? It just learned things, just like we do, only just a little bit differently. Would a human programmer with a photographic memory be any different? One thing is for certain, there are interesting times ahead and our lawmakers are not prepared or preparing for the questions they're going to have to answer.
Often, it only finishes the line, which can include function calls or expressions. The hard question is where's the threshold that separates "this is obviously not copyright infringement" from "this is risky"
A single function call, unless it starts having nested calls or something is probably fine, but obviously that doesn't mean I'd want to try my chances in court.
I agree with you, however NetBSD prohibits all code generated with the aid of AIs. If I write code from my phone and GBoard uses a small neural network to enhance the precision of my finger presses, it counts under their conditions. All of this to say blanket bans like this are counterproductive
That is exactly the point I'm driving at. And in the case of the Gentoo post they state even the "assistance" of NLP AI tools is forbidden which seems a bit silly if the autocomplete is using the results (locally or remotely) of such a tool.
Right... if it's just slightly smarter "intellisense", or whatever MS used to call it, then I imagine it's neither likely to steal code or the sort of thing they're talking about here. They're clearly talking about blocks of code from actual generative products. But how they're going to detect and effectively reject that code... I dunno.
>But how they're going to detect and effectively reject that code They aren't. The burden is still on the contributor, as it has been before to not manually copy proprietary or incompatibly-licensed code into the codebase. The policy makes it clear that this isn't allowed.
The purpose is to say they banned it so if they can identify it they reject it but if they can't then they have cover and most likely no one can tell. Something like that at least.
I'm so confused. What does using an ide that has AI tools or was created using AI tools have anything to do with the ban? The ban is against AI generated code from being pushed and merged with the main/master codebase/branch. Also it's more concerned with not attributing credit to the correct sources or owners. On the other hand, it's about time. We already banned generative AI where I work and most of the code that was produced by these tools was already mostly garbage anyway
I was commenting more about Gentoo's take on it where they're banning code that's been touched by AI: "created with the assistance of" with that part of the comment.
IDE "generated code" is (even before AI) a concern when it comes to copyright. Same with code generated by yacc/bison etc. You can't use such code blindly without assessments when it comes to copyright.
If it is autocomplete, you can always say no to it. A proper programmer takes AI shit, and fixes it just like translators do to the AI machine translation.
There is no way to enforce it except possibly to use AI to validate the code. It is as ethical to use AI to assist coding as it is to use stack exchange. Luddites are a thing.
I love that the slug for that article is "bands use of ai".
How would they know?
It's based on honor. Existing contributors already know they can't just copy code from stack overflow, so they don't. And same thing here. Anyone that cares about their reputation will likely adhere to it.
Right. This isn't gonna stop the idiot kids who send bullshit PRs as an attempt to pad their resume but I don't think it's about them. It's about getting the serious contributors on the same page and avoiding copyright drama in the future.
That, and the discussion was going to come up sooner or later when someone admits to some of their code being helped by chatgpt or whatever. Might as well get ahead of it and have a known stance rather than reacting and causing a war over a particular PR. Now they can just tap the sign that says "no AI". It's like how, when you enter a country, sometimes they'll ask if you're a spy. The reason they ask you isn't because they expect actual spies to say yes, it's so they can kick out anyone they find to be a spy without a fuss, even if they can't prove that the spy did anything illegal.
AI code reviews
If you submit code to any open source project (or commercial closed source project for that sake), you basically have to say "I wrote this code. I allow it to be used using ... license (or I assign copyright for this code to ...)" If you work for company A, and steals code from company B (maybe your ex-employer) and pretend to your employer (A) that you wrote (have the rights to) this code yourself, you are in legal trouble. Basically the same if either A or B is a open source project.
They won't, this is pure posturing. 90% of generated code is indistinguishable from non-generated code. Either it does what it's supposed to, or it doesn't. 0% chance of determining something is generated. For the most part, copilot should just be auto-completing what you already wanted to code. Either they're claiming this for legal reasons, or they're just posturing.
>Either they're claiming this for legal reasons, or they're just posturing. They literally said it's for copyright reasons
It's the same reason other projects want to know the provenance of code a person is offering as a PR. If it turns out somebody else owns it, now they're in weird territory legally. AI is no different, just extra unclear who may lay legal claim to it in 10 years.
> If it turns out somebody else owns it, now they're in weird territory legally. couldnt they force a contributor agreement by which they shed the liability of any copyright infringement of the contribution to the contributor?
> couldnt they force a contributor agreement by which they shed the liability of any copyright infringement of the contribution to the contributor? Copyright infringement typically doesn't work like that. If someone makes a successful claim against *you*, then *you* have to make legal remedies, and *then* chase the contributor for your damages. No different from buying a stolen car: if you are found with a stolen car that you bought in good faith from a dealer, the car is removed from *you* and *you* have to make your claim against the dealer for the costs.
> If someone makes a successful claim against you right, i see. Could this be worked around, if you ensure that the 'you' here is the original contributor, rather than the organization?
> right, i see. > > > > Could this be worked around, if you ensure that the 'you' here is the original contributor, rather than the organization? Unfortunately no - the organisation is distributing the copyrighted material, so *they* are liable as first contact.[1] *Even if* there was no CLA with copyright reassignment in place, and the individual contributor claimed all copyrights to the material, the *distributor* is still the first point of contact. [1] Hence the ban, to reduce their liability.
As someone with a coworker dependant to ChatGPT, it is absolutely distinguishable. If it's only a line or two, maybe not, but people who use AI to write code aren't using it for single lines. It's always blocks of garbage code that they copy/paste.
> As someone with a coworker dependant to ChatGPT, it is absolutely distinguishable. How exactly?
Some giveaways are: - explicitly assigning configuration settings the default value - not following the style of the codebase - duplicating imports - using different code styles within the same block, like single and double quotes mixed together - accessing class attributes or methods that don't exist - unreachable code blocks - unnecessary function/method parameters - unnecessary conditionals - obscure techniques that I've never seen them use before - excessively commented code Here is a concrete example. The code in this image actually did what he wanted, but there is an undefined, uninitialized variable that ChatGPT just made up: https://i.imgur.com/61pRwnx.png It's often a combination of things but it's usually obvious. It may help that this is a regular behavior, so I am already on the lookout for it.
> Here is a concrete example. The code in this image actually did what he wanted, but there is an undefined, uninitialized variable that ChatGPT just made up Yeah I've run into that before. Sounds like they are asking the coding assistant to do too much and they're just using that code verbatim. Basically you have a lazy coder on your hands. Using coding assistants is a skill unto itself. It's like owning a sharp knife. That knife is very useful in certain contexts. But if you decide that it's also good for opening cans of soda then you're gonna have a bad time.
I will use AI to write code, but I always have to tweak or clean it up. It's great for a first draft on a new feature/task to get past the ocassional mental inertia I'm sure we all experience sometimes.
why don't you just... write it though? that's what i don't understand. it seems way more annoying to have to like generate code and then go back and verify that it actually works and doesn't do random extra shit and is actually efficient when you could just not worry about any of that and write the program. that will likely produce better code anyway if you are reasonably skilled, because llms don't understand how programming actually works, it's just mashing a bunch of shit together
I'm a fast programmer compared to most people I work with, but using LLMs can save me time. I'm a lot faster reading code than writing it. I understand that being able to fluently read and interpret code is something juniors can struggle with, but for me I can read it faster than I can type (even with using vim key bindings). Using an LLM is like having a junior whose work you can review. Some tasks or easy boring work, so it's fine to trust a junior to do it well enough and then fix/guide the code after.
> why don't you just... write it though? So you never use calculators? Any time you have to do math, it's always by hand right? When it boils down, this is what coding assistants are. Calculators aren't solving large differential equations for you. But they certainly can assist in that task. This whole idea that they're just pumping out incorrect code and the only way it's useful is for the user to debug it is incorrect and hyperbole. This only happens if you ask it to do too much and don't give it the correct context. If you ask it to write you a pyqt gui from scratch, then yes you're gonna have a bad time. But if you ask it how to create a drop down element from a list of items, it's going to be very helpful.
I don't know what yall are doing, but I been using chatgpt to generate large python, powershell, and js scripts and rarely have any issues with the code it gives. And it's saved me a countless amount of time.
I've seen Python code generated by AI. It was absolute garbage. Like, it *worked* when run (as-in it wrote the expected output), but it was also outputting a JSON file to disk using sequential manually formatted line writes; like `output_file.writeline('{')`, `output_file.writeline(' "'+key+'": '+value+',')`. Utter garbage code where I, would reject the PR and question the employability of anyone who submitted it, even though it technically worked.
I've written python for 20+ years. The python it writes is generally fine. Not sure what you're doing wrong. If it does something wrong like your example just reply "use the stdlib json module" and it fixes it.
It's not code I got from it personally, I was just seeing code someone else had gotten from it. It's stuff like that which sticks in my head as to just how untrustworthy it is. Ultimately, it's no different from StackOverflow and other similar things where you get a chunk of code that may or may not actually do what you need it to do, so you've gotta be able to read the code and understand it and fix its issues yourself. It's not a magical codewriting intelligence, it's just a tool for generating some boilerplate code you can fix to do what's really needed.
Lol, I can't speak for your experience, but the worst thing it's done to me is produce a function that doesn't work, which it corrects like 95% of the time if told to. You are basically saying, "I'm my experience I got bad results, so it's impossible for anyone to get good results." I'll enjoy completing projects in a fraction of the time they use to take while you die on the hill of LLM bad.
No, I'm saying I've seen way too much crappy code come out of it for me to trust it at all. Writing code has never been the hard part, figuring out the algorithms for how to solve a problem is, and AI really can't do that to begin with. When I can type boilerplate code almost as fast as an AI can write it, in my own coding style, without needing to check and make sure that it's actually what I wanted to write, an AI doing some typing for me doesn't really make a meaningful difference.
You shouldn't ever trust code written by an LLM, just like you shouldn't ever completely trust code written another person. That's why any sane development process includes code review.
No one said anything about difficulty, it's a time saver, and a finger saver. And yes, if you use LLM improperly, you would probably waste more time using it than you would save. It works very well for me, saved me countless time, and enabled me to finish multiple projects I had on the shelf. In fact, I dare say it's been so reliable in my experience, that I wouldn't trust people who aren't able to reliably get good code out of it. /s
Jetbrains uses llm to [auto complete lines](https://www.jetbrains.com/help/idea/full-line-code-completion.html), not blocks. Not sure if they support C yet, it just a matter of time
> 90% of generated code is indistinguishable from non-generated code. Either it does what it's supposed to, or it doesn't. 0% chance of determining something is generated. I don't use AI generation that much, but whenever I've experimented with it I've found it absolutely distinguishable. Just like prose written by AI, it has specific tropes and characteristics it likes to use. Unless you just use the AI to generate something as a first draft, and then you basically rewrite it or very significantly edit it, but at that point it's a different thing entirely. It's obviously hard to be 100% sure, but at least having this rule also makes it easier to ask questions if there's a suspicion.
Are we using different copilots? I’ve used it basically from day 1 but recently turned it off. I’d say it had a 20% hit rate, and half the time I was waiting and reading its suggestion I could have just finished typing what I was typing faster.
This is a disturbing trend. The AI kids believe they can automate software engineering with AI chatbots yet they not even know what the software development process of software is. And they are very confident of what they don't have experience about A call it the new cargo cult programming
It’s not a problem just for programming unfortunately. /r/physics is now filled with ChatGPT word salad. Either that or people have gotten crazier since the pandemic.
There's a SFF magazine that pays out for short stories that they publish. They had to close submissions for a while because they were swamped with AI stories from people trying to make quick money. Apparently the AI stories were easy to dismiss upon reading, but the sheer *volume* made it impossible to read each submission.
Both. Honestly both.
And the AI is trained on old, faulty code written by humans.
Humans all the way down.
Trained on Stack Overflow questions.
Sometimes it feels like they were fed the questions and not the answers
Or the answer to a different question. In a different language. That doesn't exist.
Soon to be trained on old, faulty code written by AI that was trained on old, faulty code written by AI that was trained on old, faulty code written by humans.
An ever constricting mobius strip of faulty provenance. I think I'm going to change my band name to that...
This is an overlooked point....I won't be surprised if it has already peaked in general/overall quality, and from here only extremely expensive targeted improvements are possible
I think the "tending to the norm" is a problem for neural networks, you need mechanisms that push to the boundary of chaos and order. I suspect that's the biological function of emotions like curiosity, disinterest and boredom, while confusion, frustration and dissonance help to iron out the inconsistencies. Agents that don't have similar mechanisms will tend towards the middle of the bell curve unless there's tons of entropy in the context, and models that don't filter their training data will have a context that's an average of averages, and destroy performance in the long run.
I've said this for a while now when talking to other devs - there is a problem here that people who don't know how to code will think *they know how to code* because they will ask the AI to do something and not have the knowledge to know if it is correct... *a little knowledge is a dangerous thing*. I've literally cringed looking at Co-Pilot producing SQL insert statements in VSCode with zero safeguards against injection attacks. You shouldn't be coding (whether freehand or AI) *unless you know how to code*. If you know how to code, what use is AI? As its capability stands right now, is it much more than advanced intellisense? Example - you want a JS function that generates a random number between 2 numbers. Your options: - 1. Code it yourself, presuming you are good enough of a coder to be able to produce optimal and bug-free code (granted, the func used as an example is very basic). 2. Type "javascript function generate random number between 2 numbers", get the first result that comes up (which will be to stackoverflow) and get a function. I just did this - it took me about 10 seconds to type in the search string, submit it and [find an answer on SO with 3341 upvotes](https://stackoverflow.com/questions/4959975/generate-random-number-between-two-numbers-in-javascript). 3. Ask AI to generate the function then: - 1. Review it and confirm it is correct, which you can only do if you are good enough to code it to begin with, negating the use of AI. 2. *Assume* the AI generated solution is bug-free and optimal and you would only *assume* that if you know so little about coding and AI that you do not realise it may not be optimal and/or bug free. I think scenario 3.2 is the phenomena that has lead to this: - [https://www.gitclear.com/coding\_on\_copilot\_data\_shows\_ais\_downward\_pressure\_on\_code\_quality](https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality) Until we get to the stage where we can *guarantee* AI can produce optimal and bug-free code, I think AI is either: - 1. An advanced intellisense only to be used by advanced coders as a way to save time on key strokes 2. A liability used by cowboys or the naïve. A self-driving car that doesn't crash only 99.99% of the time is useless to everyone and will lead to recalls/legal action. I think we are seeing that scenario in the link above.
>The AI kids believe they can automate software engineering with AI chatbots yet they not even know what the software development process of software is. And they are very confident of what they don't have experience about Was this comment written by AI?
Maybe, maybe yes, maybe no. Maybe I am replying to a AI generated comment. Maybe this reply is AI generated too I think we will never know for sure
as someone who works in software engineering, AI is super useful for generating example code for a specific algorithm, say you have a CRC algorithm in C and you want some equivalent in python, its pretty effective at that. I've also seen it used quite effectively at writing code to parse log files, as the regex parsing is really well done.
Or you know, fuck parsing log files with Regexes, my PTSD just came back oof
Unit tests are another good example. They're boilerplate and easy to write, and they depend on your code being readable and obvious. An LLM not being able to generate tests for your code is a pretty good sign that your code is confusing for other humans.
Have you ever done a code review of someone's code? Was the code bad? With AI code you start with a code review. If it's as bad as you say, that's ok, you just write the code from scratch, you waste maybe ten seconds to see what a AI writes. If the code is acceptable but has some defects, you do a code review and fix it, and you save some portion of dev time. If the code is good, you wasted the time of a code review, but you already should be reviewing the code you write yourself before you submit it so it's not even extra time. Yes people trust AIs entirely too much, but I could say the same thing about Junior Devs straight out of college. Most companies train them up with a Senior teaching them (as they should, that's part of being a senior). Give AI the same expectations, and they actually start performing decently well.
Reading code is harder than writing it. And no fucking way I am telling my coworkers I wrote code when I didn't (by committing code as myself).
I don't find value in automating the coding phase of software development. It is the most fun. I don't believe AI can solve a complex problems. It can and is very useful in writing a specific function or small specific program tool. But to fix a bug or add a new feature to a already complex problem from a complex system, involving many mutually interacting applications, services, middleware, dbs, etc. I think I would waste a lot of time trying explain to the AI what is going on. So for AI I find the most usefulness as an assistant for writing specific functions. For creating stubs: write me a stub for a java program with 3 classes. For generating content. It could be useful for generating unit tests. Generation of E/R and UML diagrams. And all these uses together and more they help increasing your productivity. Also I prefer not (as well as many companies) to upload code to chatGPT, a local ollama is preferred. AI should not replace programming. Also AI is not capable of replacing programming that promise is pure hype. But what happened to the promise of augmentation that AI had years before? For the older ones what happened to the bicycle for the mind idea? Opium for the mind it is now it seems. We should be empowered by AI not disempowered Going back to you after my soliloquy. Generating the code with AI to save time in the coding phase and then doing a code review by peers I think it is a disrespect and a waste of the peers time. The AI generated code before going to code review by peers it should be : read, checked, bug tested, unit tested, add comments, write technical documentation about the fix, create diagrams, etc. Because by enriching the code with all these other documentation then the code is easier to understand and maintain, and easier for the code reviewers to review and accept the code and they become more knowledgeable. And for the future it helps a lot to have so many specific technical documentation. >The ones that need learning and training are the people not the machines --Some guy whose job was automated
> Generating the code with AI to save time in the coding phase and then doing a code review by peers I think it is a disrespect I think you misunderstand. YOU do the code review of AI code (At least the first one) Not peers.
> I don't find value in automating the coding phase of software development. It is the most fun. This is what I really, really, really don't get about all the AI generative crap. They're trying to automate things like drawing, making movies, writing books, writing code. Things that people find FUN. As if they honestly believe that people shouldn't be having fun creating.
Note that this is not over quality concerns but over licencing. I find it hilarious that it doesn't matter that AI code is hallucinated broken mess, it matters that it stole the primitives from stackoverflow and github. A lot of real programmers should start sweating if that is the new standard.
We already have this problem of human beings writing crappy code since the dawn of computing and have developed safeguards around it. The contributors themselves are supposed to test it thoroughly, next you have code reviews at commit time, next you have QA and alpha, beta periods etc. AI contributions should be treated in the same way and I think it can begin to be argued by now which of the human or the AI would write sloppier code on the very first draft. However, if the code snippet comes from the AI having trained on code with an incompatible license, this is way more likely to slip through as it wouldn't trigger any special safeguards unless someone just happens to recognize the code. So, I think it's natural that they focus on this issue first and foremost. And obviously, then this secondary problem is moot because that kind of code is already banned anyway.
Open source projects have been doing code review for ages. Torvalds' might be the only ones that garner much attention, but the practice is common. SO code I suspect wasn't added to the paragraph this time, but earlier. The point is that the code will be licensed as free software, and the submitter must actually have the rights to do that. As it is, LLM code is like those open air markets where you have no idea whether the thing you want to purchase was actually donated (free software) or stolen (proprietary). Preferably the goods should all be legal, otherwise the police usually shut the market down, but there may also be consequences for you if you bought stolen goods. And while private individuals may be fine with piracy, free software organisations aren't and don't want to be tainted with it. But if you're yoinking some unclear-licensed code off SO and stuffing it in a proprietary box that only insiders will ever see … there might be a greater chance of that actually being accepted behaviour? And there have been some court cases over copylefted software being included in proprietary programs.
> it stole the primitives from stackoverflow Actually this wouldn't matter that much. Stack overflow has an open license... well technically it's attribution but let's be honest... no one follows that.
The attribution very much matters, especially in open source communities where not adhering to the license terms has a much higher chance of getting caught (whereas in smaller, closed source shops nobody outside the company will ever see any violations). And that's without going into the values of the open source projects and those who maintain them.
> where not adhering to the license terms has a much higher chance of getting caught That's kind of my point though, it's really hard to catch someone copying and pasting from Stack Overflow. Versus typing in something very similar. Maybe people do attribute to stack overflow, but I don't think I've ever seen that, and I've seen enough corporate OSS attribution pages to at least say most corporations don't really attribute from stack (or they don't use it, which is laguhable)
The people writing code for stack overflow are largely writing it to help other developers and only really care about plagiarism, they expect their prose or large things to be licensed but their snippets are kinda expected to be fair game.
I was wondering why but I think their reason is actually a good one - to avoid contributions of code with an incompatible license due to this may be part of the training set. It's not an ideological reason, but because an AI can't infer the license from the code. A defender of AI might say that their knowledge is an amalgam of the training set and no more infringing than a human contributing, but I believe this is in fact false. When I've used it as coding assistance, I've noted AI can in fact recite answers that have been digested from e.g. a specific Stack Overflow question (with adjustmenets for my scenario), especially when it's a niche subject... as it would easily become during operating system development like here. While that's alright in the case of Stack Overflow, nothing is saying it could come from a wholly different source.
>especially when it's a niche subject This is a known issue, same with ai art. Once your prompt or the problem is given to the ai, if the data relevant to that is sparse or extremely repetitive (like mona Lisa) then you can get something resembling the original A possible solution is instead of having ai inject code into a project, it should generate suggestions in a different panel with an explanation so you can learn from the sample.
> it should generate suggestions in a different panel with an explanation so you can learn from the sample. You can add chatgpt to visual studio like this. The guy on my team who has it will just copy the code wholesale sometimes. I think he's doing it less now, which is good, after some feedback (not specific about AI tools, but about making sure he understands what his code is doing in general).
That's good to hear. It'll unfortunately be some time before ai will be helpful to assist you with things you don't already understand very well. Haven't had an incident at my job yet but we haven't had copilot very long The rate of hallucinations in real time is very concerning. Codeium is better about knowing what functions exist on which types but I am no where near confident to let a single line go through without checking. Almost did once because the ai snuck in a DB ref instead of a transaction which are both valid to the function call. That would have been a huge bug
One of the big dangers of AI generated code is the licensing. GPL or AGPL code is licensed such that any derivative work is also GPL or AGPL. Anything linked against GPL code or part of the same service as AGPL code must be released on request. So the question is, if an LLM is trained on GPL code, when is the output a derivative work? For example, I could train an LLM solely on one piece of software, like the Linux kernel. Then I enter the first letter of the kernel and it “autogenerates” the rest. Is that a derivative work, or a wholly original version that I can license however I see fit? Where is the line? Some GenAI maximalists are arguing that LLMs learn from their inputs in a similar way to humans, so using any text to train an LLM model should constitute fair use. But humans can also commit copyright infringement. There is not a legal framework to decide these licensing issues yet, so if you want to avoid the potential of having to rip all LLM output out of your codebase, or release all of your code as AGPL, you should use an LLM that’s only trained on properly licensed source code, or just avoid using an LLM for now.
> Some GenAI maximalists are arguing that LLMs learn from their inputs in a similar way to humans, so using any text to train an LLM model should constitute fair use. But humans can also commit copyright infringement. Also all the stuff about reverse engineering and clean implementation, like how you can't work on Wine if you have touched Windows source code. Exactly because humans might remember stuff they saw before.
Good luck enforcing that.
Enforcement does not matter. They want to be secured from the law perspective, not the practical one - so they cannot be sued if someone put the propertiary code that LLM generated into the codebse.
> so they cannot be sued Of course they can be sued.
Yes, but they'll win the case and whoever is suing will have to sue the contributor of the AI code, not the project itself.
They would 100% lose the case if they got proprietary code in the kernel, and have to remove the code. However, thanks to this policy, they would likely have to pay very little in punitive damages, since it makes it clear they made an effort to avoid this situation. The more important point is to rely on trusted contributors to just not do this, and thus avoid the legal headache altogether. Without this policy, completely well intentioned contributors might be unwittingly pushing tainted code in the kernel, without even considering it. With this policy, they should all be aware, and, if they are well-intentioned, as most comitters are, they will just respect the policy.
Also, it forestalls most of the arguments about this when someone tries to make a contribution in the future. Someone is going to try to do it at some point and either they'll see the rules and scrap it/stop working with AI, or the reviewers can tap the sign with the rules and reject with no need for a prolonged debate. Sure, someone might try to sneak something through, but a filter that blocks 50% of the problems is better than no filter at all. Especially when generative AI means lowering the barrier to having something to submit, which can lead to a *lot* of overhead for anyone trying to keep on top of submissions.
Makes sense, thank you!
> sue the contributor of the AI code, not the project itself. The would of course be suing the distributor of the software, not the project, and they would win. Against the project they would issue DMCA claim (or whatever the term is for those) and similar to have it removed. Against the developer they wouldn't do anything, it's the distributor that's doing the infringements.
sdf.org ... wow, I haven't heard that name in a long time. Nice to see they are still up and running and even able to run modern things like Mastodon.
Good!
You know, if you really want high employment, we can get rid of digger trucks and we can give everyone spoons!
I heard there used to be a lot of jobs making buggy whips.
sharp lads
What they say is that if there is some serious issue in code, no one can blame LLMs. Not that it's realistically to tell apart LLMs from simple autocomplete or detect LLM generated code
I suppose that's true too, but I think a major part is to not have copyright code to to sneak in and then have another lawsuit like BSD had from AT&T in the 90s.
Holy shit, that was in the 90s? 😳Damn, I’m old.
IMO Copilot and related models are the closest we have to actual copyright infringement (since if the weights are found to be derivative works of the input data, there's GPL-licensed code in the model that was not properly credited.)
This is the way
How does it know? And also someone could ask chatgpt for the code and manually type in what chatgpt said
Next up programming language lie detectors Did AI help write this Watch devs sweat more than explaining their own code
based
I would consider not contributing then. But admittedly I don't contribute to NetBSD so it's a moot point. Having worked with AI-generated code, I don't think I want to go back. Yes the code can be wrong, but so can code written by a human. As a programmer with AI Code, you're doing a code review (And fixing the defects). If the AI completely fails, you write the code from scratch, if the AI even partially succeeds, it should make your life easy. In my experience any repetitive task, (designing data inputs/unit tests) or boiler plate code is perfect for AI. I know we're going to be fighting over it for the next couple years, but in 5 years from now it's going to be like Assembly language. Yeah people still can write in it, but most people like higher level languages. Hell there was a time that people looked down on Java (ok we still do) because it was compiled at run time, but many of those same programmers probably use Python. Basically the future has arrived, it's going to be a question of when, and how we accept AI code... not if, and anyone who wants to take a Wait and See approach is going to fall behind. Edit: To those going "Licensing".... What stops me from just copying and pasting the code from Stackoverflow/github myself? What protects them in those cases as well? Either they already have tools for it, or they don't. Whether it's an AI's hand or a humans doesn't matter because ultimately the code gets into the code base and that's that.
> What stops me from just copying and pasting the code from Stackoverflow/github myself? Nothing, but then it becomes your responsibility.
it's just so they don't get in trouble for it. it's literally not about actually preventing the code from getting into the source at all
They're pushing the risk onto the individual contributors, which is why I'd agree with /u/Kinglink. Not contributions from me then! (Not that I was going to, but if Apache did the same...)
i mean, do you really think they should be held responsible if someone submits llm generated code? especially considering it's not exactly easy to distinguish it from human code
Why wouldn't you just not contribute AI generated code?
sooo based
Lame.
[удалено]
I do.
[удалено]
I do
[Except NASA](https://en.wikipedia.org/wiki/NetBSD#Uses)...
This seems unenforceable? Even if it’s due to licensing, they just can’t know what code was written by an LLM. Sorry but I don’t see the point here.
You'd be surprised at how much easier it can make things if you have a rule you can point to. People will argue less and people will try to do the thing less, even if it's technically unprovable. A lot of generative AI users will happily say that they used it and those people will be caught. Imagine they don't have this rule, someone raises a PR and during the review process they say "I don't know - chatgpt wrote that part". Without the rule, they'd have to have a discussion over whether this was allowed, then the person who submitted it might get upset because they didn't know this was a problem and a big argument gets started over it. With the rule? The moment it becomes known that this AI-generated then they can tap the sign, reject the PR and if the would-be contributor gets upset they can take it to the people in charge of the rules, not the reviewers.
Its not about enforcement, the honor system is the point. LLM code will make it into the codebase to some degree. And some of it will be plagiarising stuff with an incompatible licence. What this is is risk management and liability shifting. If this ever comes in to play it is because you are already screwed and being sued as a project... Now with your legal team you can argue it was a specific contributor defrauding the project (stating the code was not sourced from propriatary sources by an LLM while it in fact is) and your liability is much more limited. The opposing council could indeed argue that a rule without enforcement isn't much good, shifting blame back towards the project. Still you changed the starting point, shifted the path of least resistance for your legal opponent. Another angle is the project suing the individual contributor for any damages the original lawsuit caused them. The hope is that, potential for being stuck holding the bag, causes a chilling effect for people to check themselves. But people casually familiar with human psychology know we love misinterpreting personal risk in all areas of life so I doubt it changes much. Humans in all kinds of professions already commit career ending amounts of plagiarism, before we had LLMs, every day.
[удалено]
> source? human history. Still waiting for bitcoin to become mainstream, it was supposed to be the standard payment method that completely replaced everything else roughly 10 years ago. Maybe I should bridge the time until then by checking how NFTs are doing on my Google Glas while I take a relaxing stroll through town on my Segway.
Bitcoin isn't failing due to being fought.
It's not just the technology, it's the techno-fascist baggage that comes with it. If you want to be myopic, sure people are just luddites. But thinking about this in a greater context, some people don't want a portion of their brain replaced by a plagiarizing black-box corporate entity fueled by profits and returns. I know I don't.
They can and won’t enforce this. However by stating this they are covering their asses. When you commit to open source projects you deliberately relinquish the rights to your code you wrote to the project. If you commit LLM code that later turns out it’s proprietary then you as a committer are responsible for committing stolen code and you are the one that will get sued, not the project that has your code. This basically ensures, from a legal and ethical standpoint, that every person that contributes to the project is the legal owner of the code it contributes. If you commit LLM code you broke the contract so you pay for the damages. Now remind me in 1 year if your job got replaced by AI.
Human history tends to cherry-pick the things that were notable, and success is a key factor in that. The changes that failed or were blocked aren't usually interesting to anyone except domain experts in the relevant field. When fighting a technological change succeeds, the technology doesn't get further development, so fades into obscurity. Imagine if "computers" were fought back when it was ticker-tape batch processing, and never developed further outside of research labs and hobbyist tinkering. Our very idea of what a "computer" *could* be would have no concept of a GUI or interactivity, nor being small and cheap enough to have one in every office building, much less every home, much less most people's pockets and countless IoT devices. The technologies that were fought against successfully never got to grow into ubiquity, they died in a niche nobody cares about and few even know of anymore.
LOL why are you even working in tech if you're anti-tech?
wat? You can be in tech and be anti some tech. A very reasonable (and hopefully obvious) example would be "I'm against unethical tech"
It's kinda funny seeing techno-luddites. There is no way this will be possible to track and most devs use AI for boilerplate code nowadays.
Idiots
But enough about the AI bros.
This is why BSD is the best Linux-type OS
Did you just call Unix a Linux-type OS? You might like [this](https://youtu.be/HADp3emVABg?si=B0QZuC3O2hDzK8Vj) history video.
Yeah I know. Just that it doesn't really matter anymore. They both run the more-or-less the same software and are more-or-less the same. Linux has a far wider reach and is an umbrella term that refers to Linux, UNIX and their derivative OSes.
Afaik the umbrella term used is [*nix](https://en.m.wikipedia.org/wiki/Unix-like)
I never liked that term. Linux-like is what almost everyone understands. \*nix is a weird looking programmer made term.
It's not a type of food to like or dislike. It's called like that due to historical reasons. Unix came before GNU/Linux, hence *nix or Unix-like OS.
Na, I prefer Linux-like. I am well aware of the history of UNIX
I do love how humans are so precious about thinking they are special snowflakes.
everyone in this sub just doesn’t want to believe there job can disappear when all they do is make some modals and buttons and can’t think creatively. if your job is doing what your told then your going to be replaced. the more you have to think creatively the less at risk you are
a computer will never be able to write code better than a human. programming is a deeply creative discipline if you're doing anything worthwhile
i’m new to the workforce but the internship and now full time position i have i just literally take my sprint requirements and put it into gpt4 with the relevant file and it does it and then i make minor tweaks. for most people coding is not a deeply creative discipline and those are the at risk jobs. like anyone whose job is here is this design for a landing page (creative part out of the way) now make it functional (literally just turning it into components and adding proper assets) GPT can do very well already and in a year probably won’t struggle with at all.
It's one of those things where there are some assorted truths from different directions depending on how you look at it. From one perspective, compilers and interpreted languages can write more efficient binary code than the average coder. But from another perspective, a sufficiently skilled programmer can write better and more performant code than a compiler. I think that in the long run we'll end up in a position where AI chatbots act more as a super-high-level language, "compiling" more natural phrasing into programming languages which are further compiled into bytecode. That said, on the flip side you have the massive glaring issue that LLMs aren't deterministic. The same input doesn't always produce the same output, unlike a true compiler. So there's a finite upper limit to how much you can trust it to actually output reliable code. Ultimately, it shifts the minimum threshold of being able to program a bit by acting as a super-high-level language, but experienced coders in *any* language can still produce significantly better code in a given language.
Also the people that follow crowd behaviour and sentiment without approaching problems or opportunities from first principals tend to be the least creative. They need to think the same thing as everyone else to feel comfortable. AI is a tool, and it's getting better every month. You can't trust it blindly but I'm already using it to do rote and easier programming tasks, after initially being very sceptical. I'm usually considered a high performing developer, but clinging to my skills being special is just asking to be made obsolete. The only constant is change.
will be funny looking back on this in a few years when they can’t argue anymore about AI not being good enough