[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Drop 0. from the version

Our messages crossed, so I didn’t see the message where you said “versioning doesn’t need to be semantic”.

I did not say that Druid should do SemVer; I said that you should have a policy, and discuss how that policy differs from SemVer.


> On Dec 21, 2018, at 10:57 AM, Roman Leventov <> wrote:
> Julian, I mentioned semantic versioning in the previous message. The
> comparison with IntelliJ may seem shocking at first, but actually Druid may
> be semantically closer to IntelliJ than to Hadoop or Spark. Druid is a
> sever-side *application*, not a library or framework. Like Druid has
> extensions API, IntelliJ has plugin API, that is also very unstable and
> broken in every IntelliJ release, as far as I know.
> Comparison with Guava doesn't make a lot of sense - exactly because of
> that. Guava is a library, half of the Java world depends on it. Almost
> nothing depends on Druid.
> On Fri, 21 Dec 2018 at 19:46, Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>> No one has mentioned Semantic Versioning (semver [1]) yet, so I will.
>> I don't know whether the Druid developers think in terms of semver,
>> but a lot of your audience will. In semver, the shift from 0. to 1. is
>> a big event, because the "only remove APIs in major releases" rule
>> does not apply for versions < 1.0.
>> It would be good if Druid had a policy for how long APIs and will be
>> around. It does not need to be based on semver, but if it isn't, it
>> should explain how it is different than semver.
>> It should also spell out the planned release cadence. It sounds like
>> you're thinking of two major versions per year, which sounds great.
>> But note that Guava did exactly that, and got flak from a lot of
>> people because features would move from supported to deprecated to
>> gone in just six months.
>> Regarding combining release 1.0 with Druid's graduation. My gut says
>> no. A lot of people mistakenly think that graduation is a sign of
>> product maturity, whereas it's actually a sign of *community*
>> maturity. You don't want to play into those misconceptions and make
>> people think that Druid is less mature than it really is. (For the
>> record, I think Druid's product and community are both very mature.) I
>> think of 1.0 and graduation as two separate opportunities to generate
>> some news. For 1.0 you can talk about how Druid is industry strength,
>> has wide adoption at large scales; for graduation you can talk about
>> community and what that brings to governance, and adapting to market
>> needs.
>> Also regarding that "1.0" thing. How about going straight to 14.0?
>> Julian
>> [1]
>> On Fri, Dec 21, 2018 at 10:10 AM Charles Allen <crallen@xxxxxxxxxx> wrote:
>>> If I'm greedily honest, I don't want to maintain that many backport
>>> channels. I'd rather have "If you want XYZ backport for version 14, then
>>> you have to take the latest minor version for 14" and have a policy to
>>> where someone can upgrade from 14.x --> 14.latest with (hopefully) no
>>> config changes.
>>> On Fri, Dec 21, 2018 at 9:03 AM David Glasser <glasser@xxxxxxxxxxxxxxxxx
>>> wrote:
>>>> One nice advantage to moving out of 0.x is that it frees up a digit on
>> the
>>>> right side to more cleanly differentiate between "minor release (a
>> random
>>>> assortment of bug fixes, small features, etc)" and "patch release
>>>> (literally the minimum delta to give you a security fix or high impact
>> bug
>>>> fix)".
>>>> --dave
>>>> On Fri, Dec 21, 2018 at 8:58 AM Gian Merlino <gian@xxxxxxxxxx> wrote:
>>>>> I'm not too fussy around whether we do a 1.0 or simply drop the 0.
>> and
>>>> have
>>>>> it be a 14.0 or 15.0 or 16.0 or wherever we are at the time we do
>> it. I
>>>>> also like the quarterly cadence of release-from-master we had before
>> we
>>>> got
>>>>> blocked on the ASF transition, and would like to pick that back up
>> again
>>>>> (with the next branch cut from master at the end of January, since
>> we did
>>>>> the 0.13.0 branch cut in late October).
>>>>> Seems to me that good points of discussion are, what should we use
>> as the
>>>>> rule for incrementing the major version? Do we do what we've been
>> doing
>>>>> (incrementing whenever there's either an incompatible change in
>> extension
>>>>> APIs, or in query APIs, or when necessary to preserve the ability to
>>>> always
>>>>> be able to roll forward/back one major release). Or do we do
>> something
>>>> else
>>>>> (Roman seems to be suggesting dropping extension APIs from
>>>> consideration).
>>>>> And also, what does 1.0 or 14.0 or 15.0 or what-have-you mean to us?
>> Is
>>>> it
>>>>> something that should be tied to ASF graduation? Completeness of
>> vision?
>>>>> Stability of APIs or operational characteristics? Something else?
>> You are
>>>>> right that it is sort of a marketing/mentality thing, so it's an
>>>>> opportunity for us to declare that we feel Druid has reached some
>>>>> milestone. My feeling at this time is probably ASF graduation or
>>>>> completeness of vision (see my earlier mail for thoughts there) are
>> the
>>>>> ones that make most sense to me.
>>>>> On Fri, Dec 21, 2018 at 10:41 AM Charles Allen <crallen@xxxxxxxxxx>
>>>> wrote:
>>>>>> Is there any feeling in the community that the logic behind the
>>>> releases
>>>>>> needs to change?
>>>>>> If so then I think we should discuss what that release cadence
>> needs to
>>>>>> look like.
>>>>>> If not then dropping the 0. prefix is a marketing / mental item.
>> Kind
>>>> of
>>>>>> like the 3.x->4.x Linux kernel upgrade. If this is the case then
>> would
>>>> we
>>>>>> even want to go with 1.x? I think Roman's proposal would work fine
>> in
>>>>> this
>>>>>> case. Where we just call it Apache Druid 14 (or 15 or whatever it
>> is
>>>> when
>>>>>> we get there) and just keep the same logic for when we release
>> stuff,
>>>>> which
>>>>>> has been something like:
>>>>>> For a X.Y release, going to a X.? release should be very straight
>>>> forward
>>>>>> for anyone running stock Druid.
>>>>>> For a X.Y release, going to a (X+1).? or from a (X+1).? back to an
>> X.Y
>>>>>> release should be feasible. It might require running a tool
>> supported
>>>> by
>>>>>> the community.
>>>>>> For a X.Y release, going to an (X+2).? or an (X-2).? is not
>> supported.
>>>>> Some
>>>>>> things that will not have tools might have warning logs printed
>> that
>>>> the
>>>>>> functionality will change (should we change these to alerts?)
>>>>>> If this sounds reasonable then jumping straight to Apache Druid 14
>> on
>>>> the
>>>>>> first official apache release would make a lot of sense.
>>>>>> Cheers,
>>>>>> Charles Allen
>>>>>> On Thu, Dec 20, 2018 at 11:07 PM Gian Merlino <gian@xxxxxxxxxx>
>> wrote:
>>>>>>> I think it's a good point. Culturally we have been willing to
>> break
>>>>>>> extension APIs for relatively small benefits. But we have
>> generally
>>>>> been
>>>>>>> unwilling to make breaking changes on the operations side quite
>> so
>>>>>>> liberally. Also, most cluster operators don't have their own
>> custom
>>>>>>> extensions, in my experience. So it does make sense to
>> differentiate
>>>>>> them.
>>>>>>> I'm not sure how it makes sense to differentiate them, though. It
>>>> could
>>>>>> be
>>>>>>> done through the version number (only increment the major
>> version for
>>>>>>> operations breaking changes) or it could be done through an
>>>> "upgrading"
>>>>>>> guide in the documentation (increment the major version for
>>>> operations
>>>>> or
>>>>>>> extension breaking changes, but, have a guide that tells people
>> which
>>>>>>> versions have operations breaking changes to aid in upgrades).
>>>>>>> Coming back to the question in the subject of your mail: IMO, for
>>>>>>> "graduation" out of 0.x, we should talk as a community about what
>>>> that
>>>>>>> means to us. It is a milestone that on the one hand, doesn't mean
>>>> much,
>>>>>> but
>>>>>>> on the other hand, can be deeply symbolic. Some things that it
>> has
>>>>> meant
>>>>>> to
>>>>>>> other projects:
>>>>>>> 1) Production readiness. Obviously Druid is well past this. If
>> this
>>>> is
>>>>>> what
>>>>>>> dropping the 0. means, then we should do it immediately.
>>>>>>> 2) Belief that the APIs have become relatively stable. Like you
>> said,
>>>>> the
>>>>>>> extension APIs don't seem particularly close to stable, but maybe
>>>>> that's
>>>>>>> okay. However, the pace of breaking changes on the operations and
>>>> query
>>>>>>> side for non-experimental features has been relatively calm for
>> the
>>>>> past
>>>>>>> couple of years, so if we focus on that then we can make a case
>> here.
>>>>>>> 3) Completeness of vision. This one is the most interesting to
>> me. I
>>>>>>> suspect that different people in the community have different
>> visions
>>>>> for
>>>>>>> Druid. It is also the kind of project that may never truly be
>>>> complete
>>>>> in
>>>>>>> vision (in principle, the platform could become a competitive
>> data
>>>>>>> warehouse, search engine, etc, …). For what it's worth, my
>> vision of
>>>>>> Druid
>>>>>>> for the next year at least involves robust stream ingestion
>> being a
>>>>> first
>>>>>>> class ingestion method (Kafka / Kinesis indexing service style)
>> and
>>>> SQL
>>>>>>> being a first class query language. These are both, today, still
>>>>>>> experimental features. So are lookups. All of these 3 features,
>> from
>>>>>> what I
>>>>>>> can see, are quite popular amongst Druid users despite being
>>>>>> experimental.
>>>>>>> For a 'completeness of vision' based 1.0 I would want to lift
>> all of
>>>>>> those
>>>>>>> out of experimental status and, for SQL in particular, to have
>> its
>>>>>>> functionality rounded out a bit more (to support the native query
>>>>>> features
>>>>>>> it doesn't currently support, like multi-value dimensions,
>>>>> datasketches,
>>>>>>> etc).
>>>>>>> 4) Marketing / timing. Like, doing a 1.0 around the time we
>> graduate
>>>>> from
>>>>>>> the Incubator. Not sure how much this really matters, but
>> projects do
>>>>> it
>>>>>>> sometimes.
>>>>>>> Another question is, how often do we intend to rev the version?
>> At
>>>> the
>>>>>> rate
>>>>>>> we're going, we rev 2-3 major versions a year. Would we intend to
>>>> keep
>>>>>> that
>>>>>>> up, or slow it down by making more of an effort to avoid breaking
>>>>>> changes?
>>>>>>> On Thu, Dec 20, 2018 at 2:17 PM Roman Leventov <
>>>>>>> wrote:
>>>>>>>> It may also make sense to distinguish "operations" breaking
>> changes
>>>>>> from
>>>>>>>> API breaking changes. Operations breaking changes establish the
>>>>> minimum
>>>>>>>> cadence of Druid cluster upgrades, that allow rolling Druid
>>>> versions
>>>>>> back
>>>>>>>> and forward. I. e. it's related to segment format, the format
>> of
>>>> the
>>>>>> data
>>>>>>>> kept in ZooKeeper and the SQL database, or events such as
>> stopping
>>>>>>> support
>>>>>>>> of ZooKeeper for certain things (e. g. forcing using of HTTP
>>>>>>>> announcements). So Druid cluster operators cannot update Druid
>> from
>>>>>>> version
>>>>>>>> X to version Z skipping the version Y, if both Y and Z have
>> some
>>>>>>> operations
>>>>>>>> breaking changes. (Any such changes should support rollback
>> options
>>>>> at
>>>>>>>> least until the next version with operations breaking changes.)
>>>>>>>> API breaking changes are just changes in Druid extensions APIs.
>>>> Druid
>>>>>>>> cluster operators could skip any number of releases with such
>>>>> breaking
>>>>>>>> changes, as long as their extension's code is updated for the
>>>> latest
>>>>>>>> version of API.
>>>>>>>> On Thu, 20 Dec 2018 at 20:03, Roman Leventov <
>> leventov@xxxxxxxxxx>
>>>>>>> wrote:
>>>>>>>>> It doesn't seem to me that Druid API is going to stabilize
>> in the
>>>>>> near
>>>>>>>>> future (if ever), because there are so many extension points
>> and
>>>>>>>> something
>>>>>>>>> is broken in every release. On the other hand, Druid is not
>>>> Hadoop
>>>>> or
>>>>>>>>> Spark, which have applications API. Druid API for
>> extensions, not
>>>>>>>>> applications. It is used by people who are closer to Druid
>>>>>> development
>>>>>>>> and
>>>>>>>>> fixing their extensions is routine.
>>>>>>>>> With that, I think it make sense to drop "0." from the Druid
>>>>> version
>>>>>>> and
>>>>>>>>> call it Druid 14, Druid 15, etc.
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
>> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx

To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx